Tags

.NET

Beyond 2 concurrent connections in .NET
Solution · 13 Dec 2015
I’m going to document this once and for all.

The Problem

You want to an endpoint multiple times in parallel. Or maybe you want to call multiple endpoints under the same domain name. For instance, you might want to drill an API with multiple requests because the API doesn’t...
Major upgrade to Azure DocumentDB LINQ provider
Solution · 23 Sep 2015
Early Septembre 2015, Microsoft has announced a major upgrade to the LINQ Provider of the .NET SDK of DocumentDb.

I know it does appear a bit confusing since when DocumentDb was released (a year ago now), it was said that it supported SQL. Well, it supported some SQL.

Now...
DocumentDB Async Querying & Streaming
Solution · 06 Jan 2015
UPDATE (31-08-2017): This article is superseded by the new article Cosmos DB Async Querying & Streaming.

Working with the .NET client SDK of Azure DocumentDB, I couldn’t find a way to query the store asynchronously.

***This post relates to the version 0.9.1-preview of Microsoft Azure DocumentDB Client Library. If you work with another major version, this might not...
Full Outer Join with LINQ to objects
Solution · 28 Jan 2014
Quite a few times it happened to me to be looking for a way to perform a full outer join using LINQ to objects.

To give a general enough example of where it is useful, I would say ‘sync’. If you want to synchronize two collections (e.g. two collections of employees), then an outer join gives you a nice collection...
ePub Factory NuGet package
Solution · 06 Nov 2013
I’ve been publishing this NuGet package.

Ok, so why do yet another ePub library on NuGet when there are already a few?

Well, there aren’t that many actually and none are Portable Class Library (PCL).

So I’ve built an ePub library portable to both Windows 8+ & .NET 4.5.1. Why not Windows Phone? My library is based...
The Missing Windows 8 Instructional Video
Gadget · 29 Jan 2013
Scott Hanselman has produced an high quality video on YouTube.

The video is a nice and comprehensive introduction to Windows 8 for everyone (i.e. not geek only).

My experience with Windows 8 is that once you’ve figured out a few things (e.g. how to activate contextual Search), you can start appreciating the product. Before that, it just looks...
Entity Framework with Asynchronous behaviours
Solution · 13 Dec 2012
They finally did it: the future release of Entity Framework (version 6) will sport asynchronous behaviour based on .NET 4.0 Task Parallel Library (TPL).

The API is pretty neat. First the SaveChanges gets an async brother SaveChangesAsync returning a Task. So we can now write things like:

await context.SavesChangesAsync();

The more complicated topic is the queries. ...
Windows Store App Lifecycle
Solution · 24 Oct 2012
MSDN Magazine cool article about Windows Store App (aka Modern App, aka Metro Style app) lifecycle.

The author, Rachel Appel, raise the veil from how an app is managed by the OS in Windows 8.

Actually there are few states for an application to be in and few transitions but the devil is in the details!

<img title="How Windows...
Code Review in Visual Studio 2012
Solution · 22 Oct 2012
The following article discusses the code review & annotation feature of Visual Studio 2012.

Finally integrated in Visual Studio & TFS, this feature is another step forward to integrate the entire Application Lifecycle Management (ALM) in one toolset.
Convertible Tablet: HP Envy x2
Gadget · 12 Sep 2012
Finally some competition to Microsoft’s Surface hybrid!

[youtube http://www.youtube.com/watch?v=qHroxzEZfIY] This goes in another direction (compared to Surface). This means the paradigm one-OS multiple hardware vendor is still good for consumers!
Mocking Dependencies != IoC (or at least it doesn’t have to)
Solution · 05 May 2012
Since I met the concept of Inversion of Control (IoC) I’ve been a big fan! What an elegant way to externalize dependencies and decouple a component from all the components it depends on. No longer is a component initializing a dependency initially coupling itself in a hard way.

A nice side-effect is that an IoC-compliant component is easier to...
Expression Trees Blog Series
Solution · 16 Dec 2011
I’ve wrote a blog series around expression trees in .NET 4.0:
EF Power Tools CTP1 Released
Solution · 28 Jul 2011
A little while ago, Microsoft released the first CTP of Entity Framework Power Tools.

Those power tools work on top of EF 4.1 and provide the following exciting features:
- Reverse Engineer a Code First Model from an existing database. Now this comes with the caveat that everything is configured through...
Expression Trees: Part 4 – Simple mapping inversion
Solution · 25 Jun 2011
This is part of a series of blog post about expression trees in .NET 4.0. The past blog entries are:
In this article, I’ll cover simple mapping inversion using Expression Trees (ET). By simple I mean that we’ll map two objects properties / fields...
Expression Trees: Part 3 – Setting Properties
Solution · 16 Jun 2011
This is part of a series of blog post about expression trees in .NET 4.0. The past blog entries are:
- Basics (1)
- Fetching Properties (2)
In this article, I’ll cover using Expression Trees (ET) for setting properties.

I’ll use the same object model I’ve used in the last article for examples:

public...
Expression Trees: Part 2 – Fetching Properties
Solution · 13 Jun 2011
This is part of a series of blog post about expression trees in .NET 4.0. The past blog entries are:
- Basics (1)
In this article, I’ll cover using Expression Trees (ET) for fetching properties.

In this article, I’ll use the following classes:

public class Employee
{
public string FirstName { get; set;...
Expression Trees: Part 1 – Basics
Solution · 08 Jun 2011
I’m currently doing an SOA project and I have to put forward the solution architecture for many aspects of the solution. If you’ve done a bit of true SOA project (not to get into religious debate, let’s just say ‘where your architecture is oriented around services not just as a remote procedure call mechanism but where the services are first...
Entity Framework 4.1 – MSDN Article
Solution · 09 May 2011
There is a comprehensive article about Entity Framework 4.1 on MSDN Magazine of May 2011:

http://msdn.microsoft.com/en-ca/magazine/hh126815.aspx

You can also read my blog series on the topic.
Entity Framework 4.1 Series
Solution · 21 Apr 2011
I did a few Proof of Concepts (POC) around the new Entity Framework 4.1 as part of an SOA project I’m currently on and I decided to share the acquired knowledge. The following series of blog posts are the result:
Entity Framework 4.1: Bypassing EF query mapping (8)
Solution · 21 Apr 2011
This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
In this...
Entity Framework 4.1: Inheritance (7)
Solution · 19 Apr 2011
This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
In this article, I’ll cover inheritance.

…
Entity Framework 4.1: Optimistic Concurrency (6)
Solution · 17 Apr 2011
This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
In this article, I’ll cover optimistic concurrency.

Very often we need to manage many-users concurrency. This...
Entity Framework 4.1: Many to Many Relationships (5)
Solution · 15 Apr 2011
This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
In this article, I’ll cover the many to many relationships.

Let’s start with the easiest case, we’ll let EF...
Entity Framework 4.1: Complex Types (4)
Solution · 13 Apr 2011
This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
In this article, I’ll cover the complex types.

By default, EF 4.1 is mapping classes to table. That is convenient but sometimes, we...
Entity Framework 4.1 Release to Web (RTW)
Solution · 12 Apr 2011
Two weeks ago, Microsoft published a Release Candidate (RC) of Entity Framework 4.1. They promised to release the product within 4 weeks.

Well, they released it today!

So you can now use Entity Framework 4.1 which is a fully supported Microsoft Product.

The only change done from the RC is the default length of nvarchar from 128...
Entity Framework 4.1: Deep Fetch vs Lazy Load (3)
Solution · 11 Apr 2011
This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
- Basics (1)
- Override conventions (2)
In this article, I’ll cover the control of what is getting loaded in queries.

EF 4.1 is able to manage relations. Now which relations get loaded when you do a query? ...
Entity Framework 4.1: Override conventions (2)
Solution · 06 Apr 2011
This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
- Basics (1)
In this article, I’ll cover how to over conventions.

We’ve seen that EF 4.1 Code-First infer the mapping between the model (classes) and the tables according to conventions. When those conventions aren’t to our liking, we...
Entity Framework 4.1: Basics (1)
Solution · 03 Apr 2011
As I wrote a few days ago, I’ve started to toy around with Entity Framework 4.1, more specifically around the features of Code First.

I’m currently consulting for an SOA project and I have to put forward the solution architecture for many aspect of the solution, including the data access layer. So I did quite a few POCs in...
Entity Framework 4.1 RC: Including Code-First
Solution · 29 Mar 2011
Microsoft has published the first release candidate of Entity Framework 4.1. This new release includes the very popular Code First feature, allowing to model your data in the code, without any designer tools.

With this Release Candidate comes a promise: the final release will come within the next 4 weeks and will be %100 compatible with the current RC.

…
Asynchrony in .NET
Solution · 26 Mar 2011
Microsoft recently released a whitepaper on the future of Asynchrony in .NET along with a CTP SDK.

This CTP attempts to address one concern: how to make asynchrony easy in .NET by removing all the friction in code.

In .NET, Asynchrony, so far, as been treated with the pattern BeginXYZ / EndXYZ dynamic duo. For instance, in the...
Entity Framework Code First CTP 5
Solution · 22 Jan 2011
I wrote a blog post a while ago about Entity Framework’s POCO capability. Basically, the current state is that you don’t have POCO. You have a T4 template to generate classes without EF attributes or base classes, but it stops short of being a real POCO implementation once you start looking at the collections.

Microsoft is working on a...
Introduction to TPL Dataflow
Solution · 04 Jan 2011
In Octobre 2010, Microsoft released a white paper on an oncoming library in the .NET Framework, the Task Parallel Library DataFlow.

The dataflow library is built on top of the Task Parallel Library (TPL) included in .NET 4.0. Basically, the TPL provides the building block for task-oriented Frameworks. The dataflow...
Entity Framework Patterns: Select Multiple Entities
Solution · 16 Dec 2010
I just published another new article on code project:

http://www.codeproject.com/KB/linq/EFPatternsSelectMultiples.aspx

It’s still pending for reviews.

I thought I could start teasing out some Entity Framework patterns for non-trivial scenarios.

In this case, I’m trying to address the scenario where you want to select a given list of entities (you’re given the list of IDs of those entities). I...
Entity Framework 4.0: POCO or POCO?
Solution · 12 Nov 2010
The Entity Framework shipping with .NET Framework 4.0 is a major improvement over .NET Framework 3.5 SP1 Entity Framework, which was the first version. The first version was a nice curiosity but had many shortfall. For instance, it didn’t support stored procedures, the designer wasn’t flexible, the produced ad hoc queries were quite hard to read, etc. .

EF...
Scott Gu: Silverlight is strategic for Microsoft
Solution · 05 Nov 2010
The Gu has spoken: Silverlight is important and strategic for Microsoft.

A bit like I mentioned earlier this week, the main points of Microsoft strategy are that Silverlight is for rich client applications while HTML 5 is for broader reach.

An interesting point of Scott Gu is that many of...
Microsoft Silverlight Strategy
Solution · 02 Nov 2010

Apparently I wasn’t the only one to notice the lack of vigour behind the Silverlight marketing machine at the PDC last week. Indeed, that and Bob Moglia’s declarations triggered one of those cyber-tsunami internet is famous for.

Moglia, Microsoft President in charge of server and tools business, made an <a...
HTML 5 or Silverlight?
Solution · 29 Oct 2010
Ah, this is a tough question!

As I mentioned yesterday, I was in Microsoft Office in Montreal for the PDC keynote diffusion after which there was a discussion about the keynote.

The room was filled mainly with freelancers and small company consultants (that is quite typical for a Montreal crowd actually). More than half of those individuals are Microsoft...

API

Kusto Ingestion REST API
Solution · 02 Jul 2020

Yes, this week we have Kusto & a Kitty. Can’t get better than that ;)

We discussed ingestion in Azure Data Explorer / Kusto at length in past articles. We mentioned queued ingestion along the ride without diving much into it. Let’s do that now.

Queued ingestion is available in the SDK...
Accessing Azure Key Vault from within Azure API Management
Solution · 19 Nov 2019

Today we look at a common although slightly advanced scenario with API Management: accessing Azure Key Vault from Azure API Management.

In an Enterprise, API Management service are often shared between teams. This means a lot of people might open it in the Portal and look at it. It also means that putting...
Tracing request in Azure API Management
Solution · 13 Nov 2019

Azure API Management (API-M) is an API Gateway solution.

It is quite easy to create an API in API-M, connect it to a back-end API and test it in the portal.

What happen when you test it from another client platform (e.g. Postman) and it fails? You have no visibility into what...
API Management - OAuth and private back-ends
Solution · 31 Jul 2019

Azure API Management is a fully managed API Gateway service.

In my last article we looked at the anatomy of the service. Today I wanted to demonstrate how to use OAuth with JWT token to protect an API Front End. I also wanted to show how we can access backend APIs...
Anatomy of API Management
Solution · 24 Jul 2019

When I want to wrap my head around a non-trivial Azure service with a few moving parts, I like to draw a diagram.

That might be from my UML days or it might just be that I reason better with boxes that concepts spread in documentation and APIs. In general, it helps me...
How to use Azure Data Lake Storage REST API
Solution · 15 May 2019

Azure Data Lake Storage (ADLS) Generation 2 has been around for a few months now.

That new generation of Azure Data Lake Storage integrates with Azure Storage. This makes it a service available in every Azure region. It also makes it easier to access as it is built on foundation well known...
API Management exclusive access to Azure Function
Solution · 28 Mar 2019

[Update 05-04-2019: Erratum on the original article. Logic Apps is actually able to perform public IP filering.]

Azure API Management acts as a front door to your APIs.

Typically, we do not want users / apps to be able to access the underlying APIs directly since that would bypass the API Management policies,...
Accessing Azure Key Vault using Managed Service Identity Logic Apps
Solution · 13 Feb 2019

Azure Key Vault is a great service to manage secrets, keys & certificates.

It uses RBAC to control access. Like all access control system, there is a chain of access. For instance, my user account has access to the vault: this means if my account’s credentials get leaked, the access to the...
Using Azure DevOps REST API to start multiple releases with Azure Logic Apps
Solution · 12 Dec 2018

In our last article, we looked at how we could leverage the Azure DevOps REST API to trigger multiple releases.

This is useful when rebuilding an environment: first we release the shared infrastructure then each service on top of it.

In this article, we’ll see an implementation example using Azure Logic...
Using Azure DevOps REST API to start multiple releases
Solution · 05 Dec 2018

Azure DevOps CI / CD (i.e. Build / Release) mechanism is very valuable to me. It makes things reproduceable & robust. It allows me to change whatever I want since I have this constant safety net underneath.

For simple projects, having a build and release is fine. With micro-services I adopt the pattern of...
How to get rid of /api in Azure Function’s route?
Solution · 14 Feb 2018
We looked at Azure Functions.

We also looked at security around Azure Function used to implement APIs.

Something people will quickly notice when implementing an Webhook / API function is that its URL or route is always prepended by /api.

For instance, if we create a webhook...
Security with API: OAuth, token-based access vs key-based access
Solution · 11 Dec 2017
Let’s consider security with APIs, i.e how to securely identify the caller.

There are two authentication methods quite popular in the cloud to secure APIs:
1. Key-based access
2. OAuth, or token-based access in general
Let’s compare them.

Key-Based

By key-based we mean...
Azure Functions HTTP – Authorization Levels
Solution · 04 Dec 2017
In a past article, we looked at Serverless compute in Azure in general and Azure Functions specifically.

In this article we wanted to focus on Azure Function triggered by HTTP requests and the different options we have to authenticate:
- Anonymous
- Function
- Admin
- System
- User
Those are...
Nuget WordPress REST API – Demo App
Solution · 06 Dec 2016
I’ve had a few requests to explain how to use the Nuget WordPress REST API beyond authentication.

In order to do this, I added a Demo App under source control.

The Nuget package source code is available at http://wordpressrestapi.codeplex.com/ and if you download the code, you’ll see there are 3 projects:
- WordpressRestApi: essentially the Nuget package
Refactoring Tags in WordPress Blog
Solution · 17 Oct 2016
I did refactor the tags of my blog this week end!

I display the tags (as a word cloud) on the right-hand side of my pages. The tags grew organically since I started blogging in 2010.

As with many things that grow organically, it got out of hand with time. ...
Securing REST API using Azure Active Directory
Solution · 11 Mar 2016
Scenario: you have a web & mobile front-end, both using a REST API as a back-end. You want to secure that back-end with authentication / authorization. How do you do that in Azure?

There are obviously a bunch of ways to do that. In this post, I’ll discuss the recommended...
Docker Containers on Windows Server
Solution · 26 Aug 2015
If you had any doubts about the increased pace in IT innovation, look at Docker Containers. The project was open sources in March 2013 as a container technology for Linux and 1.5 years later, in Octobre 2014, Microsoft announced they were integrating that technology on Windows Server 2016!

That’s 1.5 years from toe in the water to major...
Nuget WordPress REST API - On Codeplex!
Solution · 20 Aug 2015
I’ve introduced WordPress REST API Nuget package in my last blog post.

It’s an SDK to access Wordpress REST API.

Today, I’ve open sourced it on Codeplex.

You can visit it on https://wordpressrestapi.codeplex.com/.

If you want to collaborate to increase the API...
Nuget Wordpress REST API – Authentication
Solution · 16 Aug 2015
I use Wordpress.com as my blog platform. It hosts the Wordpress CMS software and adds a few goodies.

I was curious about their API after noticing that my Blog App (Windows Live Writer) tended to create duplicate of pictures, leaving lots of unused assets...
SOA vs Mobile APIs
Solution · 02 Aug 2015
I recently read an article from Bill Appleton of Dream Factory with the provocative title SOA is not a Mobile Backend.

It raised quite a few good points that were in the back of my mind for more than a year.

Basically, what is the difference between SOA and API?

<img style="background-image:none;float:right;padding-top:0;padding-left:0;display:inline;padding-right:0;border:0;" title="inclusion-229302_640[1]"...
Azure Key Vault
Solution · 03 Feb 2015
Has somebody been peeking on my X-mas list?

Indeed, one of the weakness of the current Azure Paas solution I pointed out last year was that on non-trivial solutions you end up with plenty of secrets (e.g. user-name / password, SAS, account keys, etc.) stored insecurely in your web.config (or similar store).

I was suggesting, as a solution, to...
Surveying the Web API Security Landscape
Solution · 10 Jan 2015
Ronnie Mitra published a nice review of the current security protocols & standards available to you to protect your API.

Ronnie gives a good overview of what used to be available, in SOAP WS-*, what is currently available (HTTPS, OAuth and the like) and what is brewing for the future (Oz & Macaroons).
Description of your REST API: Swagger & RAML
Solution · 23 Dec 2014
About a year ago I did a post about WADL. I hope you didn’t build a billion dollars business model around that initiative because it hasn’t left the ground yet :(

WADL, or Web Application Description Language for the initiated, was aimed to be what WSDL is for SOAP Web Services but for REST.

You see, the thing with...
Large Projects
Solution · 18 Nov 2014
There is something about large projects that you'll never find, hence never learn, in smaller projects. The complexity, both technical and in terms of people dynamics, creates an all new set of challenges.

I read the article I Survived an ERP Implementation – Top 10 Gems of Advice I Learned the Hard Way at the beginning of the...
WADL in a bottle eating noodles
Solution · 20 Nov 2013
In my last entry about REST web services I talked about its biggest weakness for me: the lack of description model of REST services.

The idea of hitting an HTTP endpoint as a shot in the dark is for me quite a leap of faith, and very likely an invitation to spend hours troubleshooting.

But despair no more, enters...
REST style with Hypermedia APIs
Solution · 11 Nov 2013
Once upon a time there was SOAP. SOAP really was a multi-vendor response to CORBA. It even share the same type of acronym, derived from object. Objects are so 90’s dude… The S in SOAP stands for Simple by the way. Have a go at a bare WSDL and try to repeat in your head that it is...
Applied SOA Series
Solution · 01 Nov 2013
I’ve written (a year and a half ago actually) a blog series around all the concerns and issues around implementing SOA in an Enterprise Solution:
Applied SOA: Part 9–Service Versioning
Solution · 08 May 2012
This is part of a series of blog post about Applied SOA. The past blog entries are:
In this article, I’ll cover Service Versioning.

In SOA a Service...
Applied SOA: Part 8–Security
Solution · 09 Feb 2012
This is part of a series of blog post about Applied SOA. The past blog entries are:
In this article, I’ll cover security. Security is a very broad topic and by...
Applied SOA: Part 7–System Consistency
Solution · 06 Jan 2012
This is part of a series of blog post about Applied SOA. The past blog entries are:
In this article, I’ll cover system consistency. In every distributed solution, a major...
Applied SOA: Part 6–Service Composition
Solution · 21 Dec 2011
This is part of a series of blog post about Applied SOA. The past blog entries are:
In this article, I’ll quickly cover service composition.

Service Composition is often stated as a goal of SOA. I...
Integration in the Cloud
Solution · 17 Dec 2011
I ran across a series of good blog posts I wanted to share with you, dear readers.

From Richard Seroter, the series is about integration patterns in the cloud, looking at cloud-to-cloud & cloud-to-on-premise scenarios.

Richard looks at three main patterns:
For each...
Applied SOA: Part 5 – Interoperability
Solution · 13 Dec 2011
This is part of a series of blog post about Applied SOA. The past blog entries are:
In this article, I’ll cover the Interoperability. SOA aims is to expose capabilities through services in order...
Applied SOA: Part 4 – Service Taxonomy
Solution · 09 Dec 2011
This is part of a series of blog post about Applied SOA. The past blog entries are:
In this article, I’ll cover the Service Taxonomy. On an SOA project, we’re going to have services. How should we classify...
Applied SOA: Part 3 – Service Discovery Process
Solution · 29 Nov 2011
This is part of a series of blog post about Applied SOA. The past blog entries are:
- Introduction
- SOA Basics
In this article, I’ll cover the Service Discovery Process. You’re working on an SOA project. There’s going to be services. What are those services?

That sounds...
Applied SOA: Part 2 – SOA Basics
Solution · 27 Nov 2011
SoThis is part of a series of blog post about Applied SOA. The past blog entries are:
- Introduction
In this article, I’ll cover the basics of SOA.

The hardest question for an Architect to answer briefly is ‘what is architecture’. Probably the second hardest question is ‘what is SOA’? In both cases, if you receive a...
WCF Express Interop Bindings
Solution · 02 Aug 2011
Something that might come as a surprise for somebody who worked with WCF (even for years) but never had to interoperate with another platform than .NET is how much WCF isn’t interoperable out-of-the-box.

It isn’t WCF fault really. It’s SOAP’s fault and its lousy specs. Well… not so much lousy as big, complicated and extremely flexible. The result is that...
Applied SOA: Part 1 – Introduction
Solution · 17 Jun 2011
I’ve been involved in multiple projects sporting some form of Service Oriented Architecture using Microsoft .NET technologies. Some were SOA in name only (Remote Procedure Calls were involved) while others were closer in spirit to the tenets of SOA.

The term SOA tends to be abused in the community. When I do interviews with candidates, I often see SOA on...
Globalization and web services
Solution · 17 Jan 2011
I always found that globalization is the poor child of computer literature. In any .NET book, you won’t hear specifically about globalization before chapter 23. You can even tell it’s not shooting high on the radar when you look at the ways globalization was implemented in the three main front-end Frameworks of .NET: WinForm (great!), ASP.NET (retrofitted in sub-optimal way...
Globalization Patterns in WCF (WS-I18N implementation)
Solution · 31 Dec 2010
I recently came across a good Code Project article:

Globalization Patterns in WCF (WS-I18N implementation)

written by Pablo M. Cibraro.

Basically, there exists a standard specs (WS-I18N) describing how to pass international information (e.g. locale & time zone) to a SOAP endpoint in order for that web service to return you localized data.

An interesting aspect...
Using WCF for exposing and consuming OData
Solution · 10 Dec 2010
I just published a new article on CodeProject:

http://www.codeproject.com/KB/webservices/WCFDataServices.aspx

It hasn’t been reviewed yet.

In that article I do showcase different ways to use OData to expose your data using WCF Data Services. The samples grow in complexity, going from the standard hello world of WCF Data Services (ie exposing your DB on a web endpoint), to more...
Sharing Data Contracts between clients & servers with WCF Data Services
Solution · 17 Nov 2010
I’ve been blogging a bit about the OData protocol put forward by Microsoft and even wrote an article about it on Code Project. That article is supposed to be followed by others about WCF Data Services, the .NET implementation of OData, well… stay tuned!

I love...
Consuming FaultContract with Restful WCF services
Solution · 16 Jul 2010
I’ve learned something new this morning while doing a proof-of-concept around WCF REST services.

I did a little WPF demo app both hosting and consuming web services. I was trying to handle faults properly, using fault contracts. I had experience with SOAP-based WCF web services, so I went the motions, starting by creating a fault contract:

[DataContract<a...
Overview of OData on Code Project
Solution · 05 Jul 2010
I’ve contributed an entry-level article on code project about the Open Data Protocal (OData).

I go through the basics of the protocol, how it is structured, how to consume it and where I see usage.

I did this in order to follow suit with an article on ADO.NET around OData and I didn’t have anything to point at for foundation!

…

Automation

Starting / Stopping Kusto cluster with Logic App
Solution · 22 Apr 2020

In past articles, we looked at how to archive Azure Monitor data using Kusto (ADX) & how to automate that process using Azure Logic Apps.

If the sole purpose of that specific Kusto Cluster is to archive Azure Monitor Data (as it is in my case for my blog), there is no...
Automating archiving Azure Monitor Data with Kusto
Solution · 15 Apr 2020

In our last article, we laid down a method on how to archive Azure Monitor Data using Kusto (Azure Data Explorer). This allows us to later analyse that data on a much longer period than the Azure Monitor retention period.

In this article, we will automate that archiving...
API Management - OAuth and private back-ends
Solution · 31 Jul 2019

Azure API Management is a fully managed API Gateway service.

In my last article we looked at the anatomy of the service. Today I wanted to demonstrate how to use OAuth with JWT token to protect an API Front End. I also wanted to show how we can access backend APIs...
Deploying an ARM template hierarchy
Solution · 24 Apr 2019

ARM templates are a powerful way to deploy Azure resources.

It’s a declarative language which express the target state. It can therefore be used to create or update resources.

An ARM template is a JSON file. Once a target is big enough it becomes convenient to split the ARM template into multiple files....
Using Azure DevOps REST API to start multiple releases with Azure Logic Apps
Solution · 12 Dec 2018

In our last article, we looked at how we could leverage the Azure DevOps REST API to trigger multiple releases.

This is useful when rebuilding an environment: first we release the shared infrastructure then each service on top of it.

In this article, we’ll see an implementation example using Azure Logic...
Using Azure DevOps REST API to start multiple releases
Solution · 05 Dec 2018

Azure DevOps CI / CD (i.e. Build / Release) mechanism is very valuable to me. It makes things reproduceable & robust. It allows me to change whatever I want since I have this constant safety net underneath.

For simple projects, having a build and release is fine. With micro-services I adopt the pattern of...
Azure Dev Ops CI / CD pipelines with AKS - Lessons learned
Solution · 17 Oct 2018

I have this personal project where I have a swag of micro services collaborating to serve different web applications.

I started to work on the Continuous Integration / Continuous Deployment (CI / CD) using Azure DevOps.

I did toy around those areas before. But this was really a rubber hits the road experience...
Service Principal for Logic App Connector
Solution · 25 Sep 2018

Azure Logic Apps is a powerful integration platform.

It integrates with different services (inside and outside Azure) using connectors.

Connectors are responsible to authenticate to the service they represent.

Some connectors will hold the credentials. This is the case, for instance, of the SQL connector.

Other connectors will by default take the...
Deploying AKS with ARM Template – Network integration
Solution · 28 Aug 2018

In a past article, we looked at how Azure Kubernetes Services (AKS) integrated with Azure Networking.

AKS is a managed Kubernetes service in Azure.

In this article, we are going to do two things:
1. Deploy an AKS cluster with Advanced Networking using an Azure ARM Template.
2. Deploy a service on the...
Creating a Service Principal with Azure CLI
Solution · 23 Aug 2018

Service Principals are a bit of a weird beast. They are Azure Active Directory applicationswith kind of an extra bit. That bit says they can actually login by themselves. Hence the name principal. But being an application is kind of weird.

We covered Service Principals in the past. We covered how to...
RBAC and role assignment using ARM Templates
Solution · 15 Aug 2018
Azure supports Role Based Access Control (RBAC) as an access control paradigm.

It allows to map a user (or a group of users) to a role within a given scope (resource, resource group, subscription or management group).

For instance, we could map my user identity to a Virtual...
Cosmos DB Configuration Management
Solution · 20 Jun 2018
I often found the need to automate Cosmos DB deployments.

I like to have users run an ARM template and be ready to go. Automation is key component to repeatability. It enables automated testing and increase agility in many other ways.

ARM Templates don’t go beyond a database account.

…
Using Azure Container Instances to complete automation
Solution · 13 Jun 2018
We looked at Azure Container Instances.

It allows us to launch a container without a cluster and without a VM.

It is ideal for bursting scenarios.

In this article we’ll look at another scenario for container instances: automation.

The limits of ARM Templates

A great tool for automation...
Implementing & Automating Azure Stream Analytics Pipeline
Solution · 31 May 2018
In our last article, we set out to build a resilient architecture around streaming events.

In this article, we are going to build the solution. We are going to use an ARM template which automates deployment. We will also go through the configuration of different services.

The goal...
Linux Custom Script - Docker Sandbox
Solution · 11 Apr 2018
I do a lot of proof of concepts (POCs) as part of my job.

I hate keeping demo environment around. They tend to become brittle, out-of-date and filled with the last stuff I did.

I prefer to start from a clean slate every single time when possible.

That means...
Finding a VM Image Reference, Publisher & SKU
Solution · 10 Jan 2018
Automation is great.

I love to script solutions in Azure.

But most of the time, that automation started with me fumbling and trialing different approaches in the portal. Once I got something around what I wanted I’ll automate.

I suspect you do the same thing.

This is why today we’re going...
Setup for populating Cosmos DB with random data using Logic Apps
Solution · 01 Nov 2017
We recently published an article about Cosmos DB Performance with Geospatial Data.

In this article, we’re going to explain how to setup the environment in order to run those performance test.

More importantly, we believe this article is interesting on its own as it shows how to use...
How to Create a Logic App Connector in an ARM Template
Solution · 28 Oct 2017
I wanted to automate the performance test setup we discussed in the last article. This includes a Logic App calling into a Cosmos DB stored procedure.

Logic App uses connectors when connecting to other services, either within or outside of Azure. Now how do we...
How to know where a Service is Available in Azure
Solution · 14 Sep 2017
Azure has a Global Footprint of 40 regions at the time of this writing (mid-September 2017).

Not all services are available in every regions. Most aren’t in fact. Only foundational services (e.g. storage) are available everywhere.

In order to know where a service is available, we...
Creating an image with 2 Managed Disks for VM Scale Set
Solution · 23 Feb 2017
UPDATE (23-06-2017): Fabio Hara, a colleague of mine from Brazil, has published the ARM template on his GitHub. This makes it much easier to try the content of this article. Thank you Fabio!

We talked about Managed Disks, now let’s use them.

Let’s create an image from an OS + Data disk & create a Scale Set with...
Automating Azure AD
Solution · 02 Feb 2017

In the previous article, we explored how to interact (read / write) to an Azure AD tenant using Microsoft Graph API.

In the article before that, we looked at how to authenticate a user without using Azure AD web flow.

Those were motivated by a...
Azure SQL Elastic Pool – Moving databases across pools using PowerShell
Solution · 08 Jan 2017

I’ve written a bit about Azure SQL Elastic Pool lately: an overview, about ARM template and about database size.

One of the many great features of Azure SQL Elastic Pool is that like Azure SQL Database (standalone), we can change the eDTU capacity of the pool...
Azure SQL Elastic Pool – Database Size
Solution · 04 Jan 2017
I mentioned in a past article, regarding database sizes within an elastic pool:

“No policies limit an individual database to take more storage although a database maximum size can be set on a per-database basis.”

I’m going to focus on that in this article.

An Azure SQL Database resource has...
Azure SQL Elastic Pool - ARM Templates
Solution · 21 Dec 2016
In my last article, I covered Azure SQL Elastic Pool. In this one I cover how to provision it using ARM templates.

As of today (December 2016), the documentation about Azure SQL Elastic Pool provisioning via ARM templates is… not existing.

Searching for it I was able...
Primer on Azure Monitor
Solution · 27 Nov 2016

Azure Monitor is the latest evolution of a set of technologies allowing Azure resources monitoring.

I’ve written about going the extra mile to be able to analyze logs in the past.

The thing is that once our stuff is in production with tons of users hitting...
I tortured Azure in the Week-End
Solution · 07 Feb 2016
In my last blog post, I showed how to author an Azure Runbook to turn VMs up and down.

In order to test it I did put a schedule where at every hour (e.g. 1:00, 2:00, 3:00, etc.) it would start the VMs and at every pass half hour (e.g. 1:30, 2:30, 3:30, etc.).

The thing is that I...
Shutting down VMs on schedule in Azure
Solution · 05 Feb 2016
I thought it was time for a post on the quintessential automation task: shutting down VMs & starting them on a schedule.

UPDATE (Nov-27th 2016): this particular task is now available directly in the VM portal (at least the shut down part of it). This article remains interesting to look at a complete example of Azure Automation.

This is a...
Azure Runbook - A complete (simple) example
Solution · 01 Nov 2015
I meant to write about Azure Runbooks (also known as Azure Automation) for quite a while.

I had the chance to be involved in the operations of a solution I helped architect. When you get beyond trivial Azure Solution, like on premise, you’ll want to have some automations. For instance, you’ll want to:
- Clean up data at the...

Big Data

Azure Data Explorer (Kusto)
Solution · 19 Feb 2020

Let’s talk about Azure Data Explorer (ADX ) also known as Kusto.

If you ask me that is the best kept secret in Azure.

Well, it isn’t exactly a secret but most people do not know about it or if they do, they just think of it as the back-end...
Python Version in Databricks
Solution · 18 Apr 2018
In the last few months, we’ve looked at Azure Databricks:
In those articles, we used the Python SDK (also...
Import Notebooks in Databricks
Solution · 27 Feb 2018

We’ve recently looked Azure Databricks:
In most cases we did share notebooks on GitHub.

Here we wanted to show...
Azure Databricks - Parsing escaping CSV files in Spark
Solution · 07 Feb 2018
In previous weeks, we’ve looked at Azure Databricks, Azure’s managed Spark cluster service.

We then looked at Resilient Distributed Datasets (RDDs) & Spark SQL / Data Frames. We also looked at an example of more tedious transformation prior to querying using the H-1B Visa...
Azure Databricks - Transforming Data Frames in Spark
Solution · 31 Jan 2018
In previous weeks, we’ve looked at Azure Databricks, Azure’s managed Spark cluster service.

We then looked at Resilient Distributed Datasets (RDDs) & Spark SQL / Data Frames.

We wanted to look at some more Data Frames, with a bigger data set, more precisely some...
Azure Databricks - Spark SQL - Data Frames
Solution · 24 Jan 2018
We looked at Azure Databricks a few weeks ago.

Azure Databricks is a managed Apache Spark Cluster service.

More recently we looked at how to analyze a data set using Resilient Distributed Dataset (RDD). We used the Social characteristics of the Marvel Universe public dataset,...
Azure Databricks - RDD - Resilient Distributed Dataset
Solution · 17 Jan 2018
We looked at Azure Databricks a few weeks ago. Azure Databricks is a managed Apache Spark Cluster service.

In this article, we are going to look at & use a fundamental building block of Apache Spark: Resilient Distributed Dataset or RDD. We are going to use the...
Azure Databricks - Getting Started
Solution · 18 Dec 2017
Apache Spark is rising in popularity as a Big Data platform. It exists on this accelerated timeline for such an impactful technology.

Think about it:
- 2009, started as a Berkeley’s University project.
- 2010, open sourced
- 2013, donated to Apache Foundation
- 2014, becomes...
Querying tables in Azure Data Lake Analytics
Solution · 15 Feb 2016
Azure Data Lake (both Storage & Analytics) has been in public preview for a month or two.

You can get started by reading this or have a look at Azure Data Lake series for more posts on Azure Data Lake.

I thought I would kick some posts about more complex scenarios to display...
Network Access Control on an HDInsight Cluster
Solution · 26 Jan 2016
In this post, I’m gona try to show how you can use an Azure Virtual Network with a Network Security Group to control the access (at the network level) to an HDInsight cluster.

For a primer on both those technologies, please refer to my Using Network Security Groups (NSG) to secure network access to an environment post.

The...
Analyzing Web Logs with Azure Data Lake Analytics (ADLA)
Solution · 24 Jan 2016
Azure Data Lake (both Storage & Analytics) has been in public preview for a month or two.

You can get started by reading this or have a look at Azure Data Lake series for more posts on Azure Data Lake.

I thought I would kick some posts about more complex scenarios to display...
Stategic Data, Infonomics & Big Data
Solution, Strategy · 17 Jan 2016
UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.

I wanted to talk a bit about your Strategic Data & the concept of Data Lake (regardless of its implementation).

Nowaday, data is seen less and less as...
Azure Data Lake Analytics - Loading files with custom C# code
Solution · 13 Jan 2016
UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.

Azure Data Lake (both Storage & Analytics) has been in public preview for a month or two.

You can get started by reading this.

I thought I would kick some posts about more complex scenarios to...
U-SQL Cross Apply
Solution · 10 Jan 2016
UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.

Azure Data Lake (both Storage & Analytics) has been in public preview for a month or two.

You can get started by reading this.

I thought I would kick some posts about more complex scenarios...
Registering assemblies in Azure Data Lake Analytics
Solution · 06 Jan 2016
UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.

Azure Data Lake (both Storage & Analytics) has been in public preview for a month or two.

You can get started by reading this.

I thought I would kick some posts about more complex scenarios...
Azure Data Lake Analytics Quick Start
Solution · 03 Jan 2016
UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.

Azure Data Lake (both Storage & Analytics) has been in public preview for a month or two.

It already has surprisingly good documentation:
- Overview of U-SQL ; walks you through diverse scenarios, ramping you...
Checkpoints in ETL
Solution · 28 Dec 2015
UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.

Extract, Transform & Load (ETL) are so 2000’s. But then again, they are more popular than ever!

On one hand, ETL are from an era where you didn’t care too much about real-time insights or couldn’t afford it because of technical limitation. ...
HDInsight Hadoop Hive - CSV files analysis
Solution · 04 Oct 2015
Ok, on a past blog we’ve been setuping Azure HDInsight for some Hive fun.

So let’s!

Today I’ll go and analyse the data contained in multiple CSV files. Those files will be created (in Excel) but in a real-world scenario, they could be either data dump on a file server...
Azure Data Lake - Early look
Solution · 30 Sep 2015
UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.

Ok, this is a super early look at the technology. Azure Data Lake was announced yesterday (September 29th, 2015) at AzureCon (and later blogged about by Scott Gu), it will public preview at the end of...
HDInsight Hadoop Hive - Setup
Solution · 27 Sep 2015
Let’s explore Hadoop Hive, shall we?

Hive is a data warehouse infrastructure built on top of Hadoop for providing data summarization, query, and analysis.

It was originally build by Facebook as an abstraction on top of Hadoop Map Reduce and now is an open source...
Hadoop ecosystem overview
Solution · 20 Sep 2015
Have taken a look at Hadoop lately?

People who do not know Hadoop think it’s a big data technology the same way SQL Server is a technology.

But Hadoop is more of an ecosystem of different modules interacting together. This is its major strength and also the source of its major...

Business

Digital Transformation - Investing in the Core
Solution · 03 May 2018
Digital Transformation is all the rage. It’s a little like virtue: everybody is rooting for it but few can actually articulate what it is!

I recently found some nice theoretical framework which simplifies its explanation.

We will first refer to that theoretical framework. We will then articulate how it...

Containers

Ingress rules in different Kubernetes namespaces
Solution · 11 Feb 2020

In this article I want to show how an ingress controller in Kubernetes can be used to route traffic to workloads deployed in multiple namespaces.

The online doc for AKS deploys everything in the same namespace. Hence this article is a thin extension to the online doc.

The basic trick is to deploy...
Monitoring metrics in AKS
Solution · 09 May 2019

AKS has a nice integration with Azure monitor. Out of the box there are a couple of dashboards for common metrics.

What if you need to go beyond those metrics?

This is what we’re going to do in this article. I’m going to show how to get the CPU usage per container. Along...
Requests vs Limits in Kubernetes
Solution · 02 Apr 2019

Kubernetes doesn’t know what resources (i.e. CPU & memory) your container needs. That is why you must give it some hints.

If you run way under capacity and / or fairly similar pods, you do not need to do that. But if you start approaching the maximum capacity of your cluster or if you...
Testing outbound connections in AKS-Kubenet and ACI
Solution · 26 Mar 2019

What happens if a pod in AKS initiates a connection with a private endpoint? Which private IP address does the outbound connection uses?

This is relevant for a private IP inside the same VNET, a peered VNET or an IP accessible via a VPN or Express Route.

In general, the private IP of the...
AKS Auto Scaler with ARM Template
Solution · 20 Mar 2019

Azure Kubernetes Service (AKS) Auto Scaler is finally out there in public preview!

The online documentation does a great job of getting us started. In this article I wanted to get it a little further with two things. First by showing how to use ARM templates to deploy an AKS Cluster with...
Network Policies with Calico in AKS
Solution · 27 Feb 2019

Calico network plugin is finally supported within Azure Kubernetes Services (AKS).

There is a very good tutorial on the online documentation, so we won’t give a walkthrough here.

Instead we will highlight a couple of points about Network Policies in general and in AKS.

Online references

On top of the aforementioned AKS...
Flex Volume in AKS
Solution · 21 Feb 2019

I wanted to start looking at a few modules helping integrate AKS with the rest of Azure.

In a past article, we looked at Pod Identity.

This time around, we will look at Flex Volume.

Flex Volume is an integration between AKS and Key Vault. It allows keys, secrets & certificates (the...
Azure AD Pod Identity in AKS
Solution · 19 Feb 2019

I wanted to start looking at a few modules helping integrate AKS with the rest of Azure.

A big integration point is identity. For many reasons, we’ll want our pods to use service principal identities:
1. Access an Azure service supporting AAD-integration
Custom Logs on AKS & Azure Monitor
Solution · 30 Jan 2019

Let’s look at a concrete problem:
- I have containers deployed in AKS
- Those container log into custom files
- I want to analyse those logs using Azure Monitor (Log Analytics)
We’ll look at how to do that.

We leverage Azure Monitor for containers.

Custom logs in Log Analytics also is...
Dynamic disks in AKS
Solution · 15 Jan 2019

Some workloads running on Azure Kubernetes Service (AKS) requires persisting state on disk.

In general, I recommend to use external PaaS services, i.e. Azure Blob Storage, Azure SQL DB, Azure Cosmos DB, etc. . Those services take care of the stateful nature of the service, manages HA, backups, geo-replication, etc. .

Persisting state...
Designing Distributed Systems by Brendan Burns
Solution · 09 Jan 2019

In this article I wanted to talk about a book I just finished reading.

Although its title and scope is around distributed systems in general, Brendan Burns discussion is mostly using containers & Kubernetes.

Brendan Burns is currently (as of this writing in November 2018) at Microsoft. He is a cofounder of...
Understanding multiple Ingress in AKS
Solution · 28 Nov 2018

Last time, I covered the ins and outs of Ingress on AKS.

We looked at how to install nginx ingress controller and how the controller is deployed as a load balanced service. We did some URL based routing and domain name overloading.

This was in the spirit of clarifying the magic...
Understanding simple HTTP Ingress in AKS
Solution · 21 Nov 2018

We looked at Kubernetes Ingress conceptually. We looked at different use cases: URL based routing and multiple domains.

We also looked at how ingress was implemented from an AKS perspective, i.e. where traffic was routed in nodes.

In this article, I wanted to get hands on. I figured we could start...
Authoring a Helm Chart on Kubernetes / AKS - Parametrized values
Solution · 07 Nov 2018

In the last article, we authored a very simple chart.

In this article I want to show how to use one of the powerful features of Helm: values.

Values act as parameters to a chart.

Having parameters is key for dev ops as we want to deploy the same chart to different...
Authoring a Helm Chart on Kubernetes / AKS - Getting started
Solution · 31 Oct 2018

As we discussed in our last article, Helm is a great technology. I recommend using it not only to manage third party packages but also your own custom-developed solutions.

But how do you author a Helm Chart?

I found that question hard to answer. The common answer is “look at existing ones”....
5 reasons to use Helm charts in Kubernetes
Solution · 24 Oct 2018

Kubernetes is a big platform to absorb. It has a rich resource & networking model. In many ways it is a miniature version of a fully fledge cloud provider, abstracting the underlying cloud provider.

That is my excuse for postponing addressing Helm for such a long time. Until today.

I’ll look...
Azure Dev Ops CI / CD pipelines with AKS - Lessons learned
Solution · 17 Oct 2018

I have this personal project where I have a swag of micro services collaborating to serve different web applications.

I started to work on the Continuous Integration / Continuous Deployment (CI / CD) using Azure DevOps.

I did toy around those areas before. But this was really a rubber hits the road experience...
About Ingress in Azure Kubernetes Service (AKS)
Solution · 10 Oct 2018
I did a bit of experimentation with Kubernetes Ingress, more specifically NGINX, lately.

I found the concept of Ingress utterly confusing at first. It is actually relatively simple. So, I thought I would share this sense of simplicity.

This article is conceptual. There will be no code nor even Portal tour.

If you want to ramp up on Ingress in...
AKS with Kubenet vs Azure Networking plug-in
Solution · 06 Sep 2018
Update (22-03-2019): This configuration is now officially documented.

I’ve been diving into Kubernetes / AKS Networking lately. I thought I would share some of the insights I stumble upon.

We know AKS has two types of networking, basic & advanced, right?
- Basic provisions its own VNET and exposes only public IPs
Deploying AKS with ARM Template – Network integration
Solution · 28 Aug 2018

In a past article, we looked at how Azure Kubernetes Services (AKS) integrated with Azure Networking.

AKS is a managed Kubernetes service in Azure.

In this article, we are going to do two things:
1. Deploy an AKS cluster with Advanced Networking using an Azure ARM Template.
2. Deploy a service on the...
Kubernetes Services in Azure AKS - Network integration
Solution · 21 Aug 2018

Azure Kubernetes Services (AKS) is a managed Kubernetes service in Azure. Not only does it provides a managed cluster, it also integrates with Azure services.

In this article we’ll explain the integration between Kubernetes’ services and Azure Networking. We won’t give code sample though as we’ll keep that for a future article.

…
Docker multi-stage build
Solution · 11 Jul 2018
I recently came across a really nice feature of Docker Build and wanted to share it.

Building a Docker Image often requires to build binaries. In my case I needed to build a .NET core application. You might need to build a Java App or any other types of...
Cosmos DB Configuration Management
Solution · 20 Jun 2018
I often found the need to automate Cosmos DB deployments.

I like to have users run an ARM template and be ready to go. Automation is key component to repeatability. It enables automated testing and increase agility in many other ways.

ARM Templates don’t go beyond a database account.

…
Using Azure Container Instances to complete automation
Solution · 13 Jun 2018
We looked at Azure Container Instances.

It allows us to launch a container without a cluster and without a VM.

It is ideal for bursting scenarios.

In this article we’ll look at another scenario for container instances: automation.

The limits of ARM Templates

A great tool for automation...
More reliable Replica Sets in AKS - Part 2
Solution · 17 May 2018
In Part 1, we discussed how, by default, replica sets in AKS aren’t highly available. In this article, we conclude the discussion. We propose an approach to deploy highly available replica sets.

The objective of this article isn’t to propose a novel way to deploy replica sets in...
More reliable Replica Sets in AKS - Part 1
Solution · 15 May 2018
Availability is a core architecture attribute often sought after.

We’ve taken a look at Azure Managed Kubernetes Cluster (AKS) here. In this article (part 1), we’re going to experiment and prove that replica set aren’t “highly available” by default. In part 2 , we’re going to look at how...
Understanding identities in Azure AKS / Kubernetes
Solution · 10 May 2018
We’ve recently looked at Azure AKS (Kubernetes Cluster Managed Services). We’ve looked at how to create a Kubernetes Cluster with 3 lines of Azure CLI.

With this we are able to interact with the cluster & deploy containers.

There is a bit of “auto magic” happening in...
Get Started with Kubernetes (AKS) in Azure
Solution · 08 May 2018
A few weeks ago we wrote an article about how to get started with Docker in Azure.

This time we’ll do it with Kubernetes using Azure AKS Service.

What is Kubernetes?

From https://kubernetes.io/:

Kubernetes is an open-source system for automating deployment, scaling, and management of...
Azure Container Registry - Getting Started
Solution · 01 May 2018
When we looked at an overview of Container Services in Azure, we took a look at Azure Container Registry (ACR).

A Docker Registry holds Docker images in repositories. Any agent can build & push those images. Nodes running Docker containers can then pull those images.

The main...
Azure Container Instance - Getting Started
Solution · 26 Apr 2018
We’ve looked at an overview of Docker Containers in Azure. One of the services we talked about is Azure Container Instance (ACI).

ACI allows us to start a Docker Container without a cluster, without even a VM.

It is ideal for bursting, recurrent jobs and dev-test.

We...
Getting started with Docker Containers in Azure
Solution · 24 Apr 2018
Let’s get started with Docker on Azure.

We’ll play a little with Docker Containers. In our container overview we’ve done a tour of the different container managed services.

For a first hands on trial, we’ll go a little more low level and use Docker on a VM. We’ll...
Linux Custom Script - Docker Sandbox
Solution · 11 Apr 2018
I do a lot of proof of concepts (POCs) as part of my job.

I hate keeping demo environment around. They tend to become brittle, out-of-date and filled with the last stuff I did.

I prefer to start from a clean slate every single time when possible.

That means...
Overview of Docker Containers in Azure
Solution · 04 Apr 2018

Let’s talk about Containers in Azure.

In this article, we’ll cover services related to Containers in Azure. We’ll see what scenarios they each address.

Azure focuses on Docker for container. There are other container technologies out there. But Docker is by far the most popular.

Our...
Docker Containers on Windows Server
Solution · 26 Aug 2015
If you had any doubts about the increased pace in IT innovation, look at Docker Containers. The project was open sources in March 2013 as a container technology for Linux and 1.5 years later, in Octobre 2014, Microsoft announced they were integrating that technology on Windows Server 2016!

That’s 1.5 years from toe in the water to major...

Data

Change Management in a Kusto Database
Solution · 26 Aug 2020

Let’s say you’ve setup an Azure Data Explorer cluster. You’ve also setup real time ingestion pipeline. If you haven’t, take a look at the Real Time Lab I’ve put together, where you’re going to build pipelines and query data in real time.

Now this has been running for weeks, maybe months.

You...
Recursive ADLS Access Control
Solution · 29 Jul 2020

Last time we showed how to use Logic App to invoke ADLS REST APIs (list blobs specifically) using AAD identities (in this case, Azure’s Managed Service Identities, MSI).

In the past, we discussed access control in Azure Data Lake Storage (ADLS). Those intricacies are useful when accessing ADLS using Azure AD authentication.

…
Azure Data Lake Storage Logic App with Managed Identities
Solution · 22 Jul 2020

Last time we discussed some gotcha with Azure Data Lake Storage (ADLS) and access control. Those intricacies are useful when accessing ADLS using Azure AD authentication.

Unfortunately, Azure AD authentication is a little more than one year old, so a lot of tools are still using the good old storage account access keys....
Access Control in Azure Data Lake Storage
Solution · 16 Jul 2020

About a year ago I did an article about Azure Data Lake Storage (ADLS) gen 2 and how to use its REST API.

ADLS gen 2 unlocked a bunch of scenarios. It is present in every region, it enables analytics operations (e.g. renaming folders), it supports HDFS protocol, it scales, etc. .

In...
Kusto Ingestion REST API
Solution · 02 Jul 2020

Yes, this week we have Kusto & a Kitty. Can’t get better than that ;)

We discussed ingestion in Azure Data Explorer / Kusto at length in past articles. We mentioned queued ingestion along the ride without diving much into it. Let’s do that now.

Queued ingestion is available in the SDK...
Azure Databricks Overview workshop
Solution · 24 Jun 2020

I haven’t written about Databricks for quite a while (since April 2018 actually) but I had the pleasure to lead an Azure Databricks workshop with a local customer recently.

For that I prepared quite a few demos now available on GitHub.

I covered quite a few angles so I thought it would...
My ultimate ingestion process for historical data at scale with Kusto
Solution · 10 Jun 2020

In my last article, we discussed different architecture aspects of large historical data ingestion.

In this article, I want to be more prescriptive and share an approach that works well for me.

Is that the ultimate process? Of course not, that is clickbait. As we discussed at length in the previous article, different...
Ingesting historical data at scale with Kusto
Solution · 03 Jun 2020

There are many ways to ingest data in Kusto.

There is batching vs streaming, queued vs command, plugins, SDK, etc. . There is also a plethora of tools / techniques to achieve this, e.g. Azure Data Factory, LightIngest, .ingest into, etc. .

In this article I want to look...
Starting / Stopping Kusto cluster with Logic App
Solution · 22 Apr 2020

In past articles, we looked at how to archive Azure Monitor data using Kusto (ADX) & how to automate that process using Azure Logic Apps.

If the sole purpose of that specific Kusto Cluster is to archive Azure Monitor Data (as it is in my case for my blog), there is no...
Automating archiving Azure Monitor Data with Kusto
Solution · 15 Apr 2020

In our last article, we laid down a method on how to archive Azure Monitor Data using Kusto (Azure Data Explorer). This allows us to later analyse that data on a much longer period than the Azure Monitor retention period.

In this article, we will automate that archiving...
Archiving Azure Monitor Data with Kusto
Solution · 08 Apr 2020
Around the turn of the year, I moved my blog to GitHub pages. GitHub pages does only the static content serving part. I surrounded it with different Azure services to have a complete blogging solution.

One of those services is Azure Application Insights for web analytics.

App Insights keeps the data...
Aggregating events with Logic App
Solution · 01 Apr 2020

Event-based processing is getting more and more popular. It’s a great way to loosely couple processes together.

An example in the data realm would be to have one ELT / ETL process finishing by publishing an event so another process (or processes) can start.

Now, how would we implement a process that requires the...
Synchronizing two streams with Kusto
Solution · 18 Mar 2020

We discussed Azure Data Explorer (ADX) and its query language Kusto in a past article.

In this article I want to talk about a typical problem in real time analytics: synchronizing two streams of data.

This happens all the time when sensor data are produced by different devices. Different devices may record measurements...
Exploring a data set with Kusto
Solution · 11 Mar 2020

We discussed Azure Data Explorer (ADX) and its query language Kusto in a past article.

In this article, I would like to look at a simple exploration scenario.

We’ll download csv files from the web, put them in an Azure Storage Account and from there, we’ll do everything in Azure Data Explorer and...
Monitoring query performance in Cosmos DB
Solution · 04 Mar 2020

A common question with Cosmos DB is: how many Request Units (RUs) should I be using?

Although it’s easy to quickly guess what you need to start, it’s not so trivial to determine exactly what is needed.

My general advice (with Cosmos DB & other DB as a service) is to provision something...
Impersonating user in ADLS with Kusto
Solution · 27 Feb 2020

We discussed Azure Data Explorer (ADX) and its query language Kusto in a past article.

In this article I wanted to show how to access an ADLS gen 2 using user impersonation, i.e. using the identity of the user running the queries to access the storage.

It would be all trivial and boring...
Azure Data Explorer (Kusto)
Solution · 19 Feb 2020

Let’s talk about Azure Data Explorer (ADX ) also known as Kusto.

If you ask me that is the best kept secret in Azure.

Well, it isn’t exactly a secret but most people do not know about it or if they do, they just think of it as the back-end...
How to use Azure Data Lake Storage REST API
Solution · 15 May 2019

Azure Data Lake Storage (ADLS) Generation 2 has been around for a few months now.

That new generation of Azure Data Lake Storage integrates with Azure Storage. This makes it a service available in every Azure region. It also makes it easier to access as it is built on foundation well known...
Monitoring metrics in AKS
Solution · 09 May 2019

AKS has a nice integration with Azure monitor. Out of the box there are a couple of dashboards for common metrics.

What if you need to go beyond those metrics?

This is what we’re going to do in this article. I’m going to show how to get the CPU usage per container. Along...
Creating a Backup Solution for Cosmos DB using change feed
Solution · 13 Mar 2019

Cosmos DB is Azure native NoSQL database. It has awesome capabilities such as global-distribution, exceptionally high availability, throughput scalability, and much, much more.

As with most NoSQL horizontally scalable databases it doesn’t have the same backup capabilities as mainstream RDBMS systems.

Cosmos DB has an automated backup...
Cosmos DB Stored Procedures - handling continuation
Solution · 27 Jun 2018
I’ve recently did some work involving Stored Procedures in Cosmos DB.

There are a few techniques to learn when our stored procedures handle large data sets. It is all about continuation.

In this article, we’ll look at how to build a robust and scalable stored procedure.

We’ll start with...
Event Hubs ingestion performance and throughput
Solution · 05 Jun 2018
Azure Event Hubs is a data streaming platform as a Service. It is an ingestion service.

We’ve looked at Event Hubs as the ingestion end of Azure Stream Analytics in two recent articles (here & here).

Here we look at client side performance of different...
Implementing & Automating Azure Stream Analytics Pipeline
Solution · 31 May 2018
In our last article, we set out to build a resilient architecture around streaming events.

In this article, we are going to build the solution. We are going to use an ARM template which automates deployment. We will also go through the configuration of different services.

The goal...
Taming the fire hose: Azure Stream Analytics
Solution · 22 May 2018
We came upon an interesting challenge recently.

Let’s have a stream of events pouring in. The source is unimportant. Hundreds, peeking at thousands, of events per second.

We need to update a table in Azure SQL with a summary of the events.

Let’s just assume the events are about...
Python Version in Databricks
Solution · 18 Apr 2018
In the last few months, we’ve looked at Azure Databricks:
In those articles, we used the Python SDK (also...
Import Notebooks in Databricks
Solution · 27 Feb 2018

We’ve recently looked Azure Databricks:
In most cases we did share notebooks on GitHub.

Here we wanted to show...
Azure Databricks - Parsing escaping CSV files in Spark
Solution · 07 Feb 2018
In previous weeks, we’ve looked at Azure Databricks, Azure’s managed Spark cluster service.

We then looked at Resilient Distributed Datasets (RDDs) & Spark SQL / Data Frames. We also looked at an example of more tedious transformation prior to querying using the H-1B Visa...
Azure Databricks - Transforming Data Frames in Spark
Solution · 31 Jan 2018
In previous weeks, we’ve looked at Azure Databricks, Azure’s managed Spark cluster service.

We then looked at Resilient Distributed Datasets (RDDs) & Spark SQL / Data Frames.

We wanted to look at some more Data Frames, with a bigger data set, more precisely some...
Azure Databricks - Spark SQL - Data Frames
Solution · 24 Jan 2018
We looked at Azure Databricks a few weeks ago.

Azure Databricks is a managed Apache Spark Cluster service.

More recently we looked at how to analyze a data set using Resilient Distributed Dataset (RDD). We used the Social characteristics of the Marvel Universe public dataset,...
Azure Databricks - RDD - Resilient Distributed Dataset
Solution · 17 Jan 2018
We looked at Azure Databricks a few weeks ago. Azure Databricks is a managed Apache Spark Cluster service.

In this article, we are going to look at & use a fundamental building block of Apache Spark: Resilient Distributed Dataset or RDD. We are going to use the...
Azure Databricks - Getting Started
Solution · 18 Dec 2017
Apache Spark is rising in popularity as a Big Data platform. It exists on this accelerated timeline for such an impactful technology.

Think about it:
- 2009, started as a Berkeley’s University project.
- 2010, open sourced
- 2013, donated to Apache Foundation
- 2014, becomes...
Disaster Recovery with VM Scale Sets & Geo-Replicated DBs
Solution · 09 Nov 2017
Last year we posted an article about different options available in Azure to implement a disaster recovery strategy.

We strongly suggest to review that article as it gives good insights about what a disaster recovery strategy is within an already resilient Cloud Environment but also clear out a...
Setup for populating Cosmos DB with random data using Logic Apps
Solution · 01 Nov 2017
We recently published an article about Cosmos DB Performance with Geospatial Data.

In this article, we’re going to explain how to setup the environment in order to run those performance test.

More importantly, we believe this article is interesting on its own as it shows how to use...
Cosmos DB Performance with Geospatial Data
Solution · 25 Oct 2017
Time for some performance tests on Cosmos DB around Geospatial coordinates!

Let’s hurt the service and see where it shines, where it has a hard time and how scaling it (i.e. increasing Request Units or RUs) helps.

We’ll publish an how-to create the same setup in a future article...
Hacking: accessing a graph in Cosmos DB with SQL / DocumentDB API
Solution · 05 Sep 2017
Azure Cosmos DB is Microsoft’s globally distributed multi-model database service.

At this point in time (August 2017) there are four supported models: DocumentDB (also named SQL because the query language is similar to T-SQL), MongoDB, Tabular & Gremlin.

We’ve seen how to use Cosmos DB with...
Cosmos DB Async Querying & Streaming
Solution · 31 Aug 2017
I wrote an article back in January 2015 about async querying Azure DocumentDB using the .NET SDK.

The service was still in preview back then.

Since then DocumentDB has been superseded by Azure Cosmos DB and the SDK has changed a bit so I thought I would...
Hyperspheres & the curse of dimensionality
Solution · 21 Jun 2017
$fractal-1118515_640$ I previously talked about the curse of dimensionality (more than 2 years ago) related to Machine Learning.

Here I wanted to discuss it in more depth and dive into the mathematics of it.

High dimensions might sound like Physics’ string theory where our...
Extended Outage @ Instapaper - Resiliency example
Solution · 12 Feb 2017
I use Instapaper extensively to store the continuous flow of internet articles I want to read. I created a bunch of tools integrating with it (e.g. monitoring atom feeds and sending new articles to Instapaper)

Last week my tools didn’t work for a while so I finally logged...
Azure SQL Elastic Pool – Moving databases across pools using PowerShell
Solution · 08 Jan 2017

I’ve written a bit about Azure SQL Elastic Pool lately: an overview, about ARM template and about database size.

One of the many great features of Azure SQL Elastic Pool is that like Azure SQL Database (standalone), we can change the eDTU capacity of the pool...
Azure SQL Elastic Pool – Database Size
Solution · 04 Jan 2017
I mentioned in a past article, regarding database sizes within an elastic pool:

“No policies limit an individual database to take more storage although a database maximum size can be set on a per-database basis.”

I’m going to focus on that in this article.

An Azure SQL Database resource has...
Azure SQL Elastic Pool - ARM Templates
Solution · 21 Dec 2016
In my last article, I covered Azure SQL Elastic Pool. In this one I cover how to provision it using ARM templates.

As of today (December 2016), the documentation about Azure SQL Elastic Pool provisioning via ARM templates is… not existing.

Searching for it I was able...
Azure SQL Elastic Pool Overview
Solution · 18 Dec 2016
What is Azure SQL Elastic Pool and what it brings to Azure SQL Database, the SQL Azure Platform as a Service (PaaS).

Traditional model

Let’s look at how Azure SQL works without elastic pools first.

Azure SQL Database comes with...
Primer on Azure Monitor
Solution · 27 Nov 2016

Azure Monitor is the latest evolution of a set of technologies allowing Azure resources monitoring.

I’ve written about going the extra mile to be able to analyze logs in the past.

The thing is that once our stuff is in production with tons of users hitting...
How does Azure Data Warehouse scale?
Solution · 31 Jul 2016
I’ve been diving in the fantastical world of Azure Data Warehouse (ADW) in the last couple of days.

I’ve been reading through all the documentation on Azure.com. If you are serious about mastering that service I advise you do the same: it is a worthy read.

In this article, I wanted to summarize a few concepts...
DocumentDB protocol support for MongoDB
Solution · 20 Jun 2016
Microsoft announced, in the wake of many DocumentDB announcement, that DocumentDB would support MongoDB protocol.

What does that mean?

It means you can now swap a DocumentDB for a MongoDB and the client (e.g. your web application) will work the same.

This is huge.

It...
How to do Data Science
Solution · 17 Apr 2016

These days, it’s all about Data Science.

What is Data Science?

Last month Brandon Rohrer, from the Cortana Intelligence and Machine Learning Blog, came up with an excellent post.

How to do Data Science?

The post basically goes over the workflow I reproduced at the right here.

I found...
What is Statistics and why should you care?
Solution · 13 Apr 2016
Unless you graduated in art, chances are you did a course in Statistics.

Chances are you hated it.

Most people I know postponed that course until the end of their degree, didn’t understand much about it and hated it dearly.

I didn’t like it either and understood very little.

A few...
Power BI Embedded in Azure
Solution · 04 Apr 2016
One of the many announces of Build 2016 (last week) is Power BI Embedded.

On the surface you might wonder what’s the difference between that and Power BI Service itself?

Power BI Service is a powerful tool. It allows you to author reports & dashboard in Power BI desktop and export them in Power BI...
How to copy and backup SQL Databases in Azure
Solution · 10 Feb 2016
Let’s say you want a copy of your database for some reasons. Scenarios might vary. Maybe you want to bring back production DB to another environment.

There are at least three ways I could think to do that with the Azure Portal:
- Copy the Database
- Restore a version of the Database
- Backup / Restore the current Database
Copy...
Row Level Security in 3-tiers applications
Solution · 09 Feb 2016
Row-Level Security is a great new feature in SQL. It is already available in Azure SQL Database and will be available in SQL Server 2016.

This is Cloud-First for you.

Row-Level Security allows you to forbid a user to see and/or access data they shouldn’t be able to see. Traditionally this is implemented at the application level, for...
Stategic Data, Infonomics & Big Data
Solution, Strategy · 17 Jan 2016
UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.

I wanted to talk a bit about your Strategic Data & the concept of Data Lake (regardless of its implementation).

Nowaday, data is seen less and less as...
Checkpoints in ETL
Solution · 28 Dec 2015
UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.

Extract, Transform & Load (ETL) are so 2000’s. But then again, they are more popular than ever!

On one hand, ETL are from an era where you didn’t care too much about real-time insights or couldn’t afford it because of technical limitation. ...
Azure Key Vault & SQL Server Connector Update
Solution · 07 Oct 2015
Azure Key Vault is alive and well!

The Azure service allowing you to store keys and secrets in a secured container has been released at the end of summer and it continues to improve.

The SQL Server Connector is a component that can be installed on SQL Server...
HDInsight Hadoop Hive - CSV files analysis
Solution · 04 Oct 2015
Ok, on a past blog we’ve been setuping Azure HDInsight for some Hive fun.

So let’s!

Today I’ll go and analyse the data contained in multiple CSV files. Those files will be created (in Excel) but in a real-world scenario, they could be either data dump on a file server...
HDInsight Hadoop Hive - Setup
Solution · 27 Sep 2015
Let’s explore Hadoop Hive, shall we?

Hive is a data warehouse infrastructure built on top of Hadoop for providing data summarization, query, and analysis.

It was originally build by Facebook as an abstraction on top of Hadoop Map Reduce and now is an open source...
Hadoop ecosystem overview
Solution · 20 Sep 2015
Have taken a look at Hadoop lately?

People who do not know Hadoop think it’s a big data technology the same way SQL Server is a technology.

But Hadoop is more of an ecosystem of different modules interacting together. This is its major strength and also the source of its major...
SQL Server In-Memory value and use cases
Solution · 13 Sep 2015

Microsoft SQL Server 2014 has made it in the leader category of Forrester’s recent In-Memory Database Platforms, Q3 2015 report.

SQL Server 2014 was behind SAP’s Hana, Oracle TimesTen & IBM DB2 with BLU acceleration. It was on pare with Teradata Intelligent Memory. Those were the leaders. Following were...
Analysing Application Logs with DocumentDb
Solution · 06 Sep 2015
Azure DocumentDB is Microsoft Document-centric NoSQL offering in the cloud.

I’ve been working with it since Septembre 2014 and I wanted to share a use case I found it really good at: log analysis.

Now that takes some context.

I have been working for a customer using Microsoft Azure...
Azure SQL Datawarehouse
Solution · 26 Jul 2015
Documentation on Azure SQL Datawarehouse, the new Azure Datawarehouse managed service, is quite thin.

The online documentation, as of today (24/07/2015), consists of 3 videos and a blog post.

Here is what I gathered.

One of the great characteristic of the offering is the separate Storage...
SQL Server 2016
Solution · 22 Jul 2015
Here’s a rundown of my favourite new features in SQL Server 2016, largely inspired by the SQL Server Evolution video.

Impact of Cloud-First on SQL Design

This is a really nice illustration of the consequences of Cloud-First for Microsoft products.

SQL has been basically flipped around as a product. ...
Machine Learning - An Introduction - Part 2
Solution · 12 Jul 2015
In a past blog entry I gave an overview of what Machine Learning is. I showed a simple linear regression example. The goal really was to explain to newcomer to the field what Machine Learning is, what type of problem it tries to solve and what the general approach is.

Of course, I used an extremely simple example in...
Machine Learning - An introduction - Part 1
Solution · 02 Jul 2015
As I mentioned before, I did specialize (through graduated studies) in Machine Learning only to drop the field after a few years of trial on the Marketplace. I felt the field wasn’t ready for prime industrial applications.

Years have past, the field has matured and now is an exciting time to...
Azure Data Factory Editor (ADF Editor)
Solution · 25 Jun 2015
Azure Data Factory is still in preview but obviously has a committed team behind it.

When I looked at the Service when the preview was made available in last Novembre, the first thing that stroke me was the lack of editor, of designing surface. Instead, you had to configure...
Azure DocumentDB: first use cases
Solution · 08 Sep 2014
A few weeks ago Microsoft released (in preview mode) its new NoSQL Database: DocumentDB.

Not Only SQL (NoSQL) databases are typically segmented in the following categories: Key-Value (e.g. Azure Table Storage, Redis), Column (e.g. HBase, Cassandra), Document (e.g. CouchDB, MongoDB) & Graph. By its name but mostly by its feature set, DocumentDB falls in the document category.

…
Entity Framework with Asynchronous behaviours
Solution · 13 Dec 2012
They finally did it: the future release of Entity Framework (version 6) will sport asynchronous behaviour based on .NET 4.0 Task Parallel Library (TPL).

The API is pretty neat. First the SaveChanges gets an async brother SaveChangesAsync returning a Task. So we can now write things like:

await context.SavesChangesAsync();

The more complicated topic is the queries. ...
Windows Azure SQL Database named an Enterprise Cloud Database Leader by Forrester Research
Solution · 29 Nov 2012
Three weeks ago Forrester released a paper on Cloud Databases.

As pointed out by Microsoft, Forrester declared SQL Azure and Amazon Relational Database Service (RDS) and Amazon DynamoDB and salesforce.com’s Database.com as leader of the pack.

That is quite impressive given the relatively late start Microsoft took on those competitors.

SQL Azure is a leading service in the...
Delaying action at transaction commit
Solution · 28 Sep 2012
I am designing a back-end where Database contention is the number-one issue for scalability.

We use those darn distributed transactions and therefore I do not know how long the transaction my component is participating into will last. If a transaction remains open for a few seconds, many of those would create contention in the Database my component is using. This...
A Hitchhiker's Guide to StreamInsight 2.1 Queries
Solution · 06 Aug 2012
Microsoft has released a very interesting tutorial (in PDF format) with an accompanying VS Solution.
StreamInsight in the Cloud: Meet Project Codename “Austin”
Solution · 09 Aug 2011
Last week I’ve talked about Microsoft StreamInsight, an engine to do real time data analysis and a nice article overviewing at just the depth (for me anyway).

Well, StreamInsight might one day even be available in the cloud with Project Codename “Austin”.

Already available as a private CTP, this product will be a nice addition to...
Microsoft StreamInsight Intro
Solution · 30 Jul 2011
Microsoft released Microsoft StreamInsight with SQL Server 2008 R2.

This is an engine for real time data analysis. It’s an alternative to traditional BI which typically crunch replicated data. Instead, StreamInsight let’s you analyse the data as it comes to you (as a stream).

The programming model is built on top of LINQ with the twist that the...
SQL Azure & ACID Transactions: not so much in back to 2001
Solution · 01 Apr 2011
I wrote a blog entry back in November 2010 about ACID Transactions & SQL Azure.

I assumed light transactions using System.Transactions.TransactionScope were not supported based on an article I’ve just read back then. The article suggested using explicit transaction management (i.e. BeginTransaction, Commit & Rollback on the returned SqlTransaction object) which I found quite...
SQL Azure Federation: Automating your database partitioning (sharding)
Solution · 05 Feb 2011
Microsoft is about to introduce the federation concept into SQL Azure. You can read about it on Microsoft SQL Azure Program Manager Cihan Biyikoglu’s blog:
- Building Scalable Database Solution with SQL Azure - Introducing Federation in SQL Azure
- How to scale out an app with SQL Azure Federations… The Quintessential Sales DB with Customer and...
7 Things You Need To Know about SQL Azure Reporting
Solution · 07 Jan 2011
The SQL Azure Team had a quite interesting blog back in mid-November around the newly released SQL Azure Reporting. Here’s a summary of their nuggets about the new technology.
1. It’s Based on SQL Server Reporting Services (SSRS)
  SQL Azure Reporting provides a subset of the features of...
SQL Azure & ACID Transactions: back to 2001
Solution · 29 Nov 2010
I meant to write about this since I read about it a little back in July, today is the day.

You know I love Microsoft SQL Azure.

The technology impressed me when it was released. Until then Azure contained only Azure storage. Azure Storage is a...
Forrester: SQL Azure Raises The Bar On Cloud Databases
Solution · 29 Nov 2010
November 2nd 2010, Forester Research published a report around Microsoft SQL Azure. The report can be found on Microsoft web site:

http://www.microsoft.com/presspass/itanalyst/docs/11-02-10SQLAzure.PDF

Basically, they interviewed 26 companies using the technology and concluded that:
- SQL Azure is reliable
- It delivers for small to medium scenarios
Querying Azure Storage
Solution · 04 Aug 2010
I’ve been reading the Wikipedia article about CouchDB lately. It always impresses me how much engineering effort and men-hours are put into so many competing platforms. I guess the least we could do is to look at the architecture of those initiatives and get the best out of them.

<img style="display:inline;margin-left:0;margin-right:0;" alt="CouchDB - Relax" align="right" src="http://couchdb.apache.org/img/couchdb-logo.png" width="400" height="110"...
Robust Non-ACID Transaction pattern in Azure
Solution · 05 Jul 2010
I’ve been working some more on some detailed design of the Azure Pub-Sub CodePlex project. This project aims at creating a pub-sub messaging infrastructure running on Windows Azure & using Azure Storage.

Azure storage doesn’t support ACID transactions in general. Each operation is a transaction within a given partition, but you can’t span a transaction on many operations. Now...

DevOps

Deploying an ARM template hierarchy
Solution · 24 Apr 2019

ARM templates are a powerful way to deploy Azure resources.

It’s a declarative language which express the target state. It can therefore be used to create or update resources.

An ARM template is a JSON file. Once a target is big enough it becomes convenient to split the ARM template into multiple files....
Azure Application Insights Role Name in web page
Solution · 06 Feb 2019

We discussed using Role Name to separate telemetries from different applications in a previous article.

We discuss the Nuget package I developed to set the role name for each telemetry in a backend application.

In this article I wanted to cover another element that receives little coverage: how to do that for...
Azure Application Insights Role Name Nuget Package
Solution · 23 Jan 2019

I’ve been using Azure Application Insights to monitor micro-services.

Since I have many micro-services, it isn’t practical to have an Application Insight resource per service. Given that those micro-services interact with each other and I want to monitor those interactions, it makes sense to get all telemetry in one place.

The issue with...
Using Azure DevOps REST API to start multiple releases with Azure Logic Apps
Solution · 12 Dec 2018

In our last article, we looked at how we could leverage the Azure DevOps REST API to trigger multiple releases.

This is useful when rebuilding an environment: first we release the shared infrastructure then each service on top of it.

In this article, we’ll see an implementation example using Azure Logic...
Using Azure DevOps REST API to start multiple releases
Solution · 05 Dec 2018

Azure DevOps CI / CD (i.e. Build / Release) mechanism is very valuable to me. It makes things reproduceable & robust. It allows me to change whatever I want since I have this constant safety net underneath.

For simple projects, having a build and release is fine. With micro-services I adopt the pattern of...
Azure DevOps - Task output to a variable
Solution · 14 Nov 2018

I’ve been building a couple of Continuous Integration / Continuous Delivery (CI / CD) pipelines on Azure DevOps lately.

Azure DevOps is formerly known as Visual Studio Team Service (VSTS). As I mentioned recently, VSTS had nothing to do with Visual Studio and we’re very happy with the new brand name.

With...
Authoring a Helm Chart on Kubernetes / AKS - Parametrized values
Solution · 07 Nov 2018

In the last article, we authored a very simple chart.

In this article I want to show how to use one of the powerful features of Helm: values.

Values act as parameters to a chart.

Having parameters is key for dev ops as we want to deploy the same chart to different...
Authoring a Helm Chart on Kubernetes / AKS - Getting started
Solution · 31 Oct 2018

As we discussed in our last article, Helm is a great technology. I recommend using it not only to manage third party packages but also your own custom-developed solutions.

But how do you author a Helm Chart?

I found that question hard to answer. The common answer is “look at existing ones”....
5 reasons to use Helm charts in Kubernetes
Solution · 24 Oct 2018

Kubernetes is a big platform to absorb. It has a rich resource & networking model. In many ways it is a miniature version of a fully fledge cloud provider, abstracting the underlying cloud provider.

That is my excuse for postponing addressing Helm for such a long time. Until today.

I’ll look...
Azure Dev Ops CI / CD pipelines with AKS - Lessons learned
Solution · 17 Oct 2018

I have this personal project where I have a swag of micro services collaborating to serve different web applications.

I started to work on the Continuous Integration / Continuous Deployment (CI / CD) using Azure DevOps.

I did toy around those areas before. But this was really a rubber hits the road experience...

Identity

Recursive ADLS Access Control
Solution · 29 Jul 2020

Last time we showed how to use Logic App to invoke ADLS REST APIs (list blobs specifically) using AAD identities (in this case, Azure’s Managed Service Identities, MSI).

In the past, we discussed access control in Azure Data Lake Storage (ADLS). Those intricacies are useful when accessing ADLS using Azure AD authentication.

…
Azure Data Lake Storage Logic App with Managed Identities
Solution · 22 Jul 2020

Last time we discussed some gotcha with Azure Data Lake Storage (ADLS) and access control. Those intricacies are useful when accessing ADLS using Azure AD authentication.

Unfortunately, Azure AD authentication is a little more than one year old, so a lot of tools are still using the good old storage account access keys....
Network Policies with Calico in AKS
Solution · 27 Feb 2019

Calico network plugin is finally supported within Azure Kubernetes Services (AKS).

There is a very good tutorial on the online documentation, so we won’t give a walkthrough here.

Instead we will highlight a couple of points about Network Policies in general and in AKS.

Online references

On top of the aforementioned AKS...
Azure AD Pod Identity in AKS
Solution · 19 Feb 2019

I wanted to start looking at a few modules helping integrate AKS with the rest of Azure.

A big integration point is identity. For many reasons, we’ll want our pods to use service principal identities:
1. Access an Azure service supporting AAD-integration
Accessing Azure Key Vault using Managed Service Identity Logic Apps
Solution · 13 Feb 2019

Azure Key Vault is a great service to manage secrets, keys & certificates.

It uses RBAC to control access. Like all access control system, there is a chain of access. For instance, my user account has access to the vault: this means if my account’s credentials get leaked, the access to the...
Service Principal for Logic App Connector
Solution · 25 Sep 2018

Azure Logic Apps is a powerful integration platform.

It integrates with different services (inside and outside Azure) using connectors.

Connectors are responsible to authenticate to the service they represent.

Some connectors will hold the credentials. This is the case, for instance, of the SQL connector.

Other connectors will by default take the...
Creating a Service Principal with Azure CLI
Solution · 23 Aug 2018

Service Principals are a bit of a weird beast. They are Azure Active Directory applicationswith kind of an extra bit. That bit says they can actually login by themselves. Hence the name principal. But being an application is kind of weird.

We covered Service Principals in the past. We covered how to...
RBAC and role assignment using ARM Templates
Solution · 15 Aug 2018
Azure supports Role Based Access Control (RBAC) as an access control paradigm.

It allows to map a user (or a group of users) to a role within a given scope (resource, resource group, subscription or management group).

For instance, we could map my user identity to a Virtual...
Understanding identities in Azure AKS / Kubernetes
Solution · 10 May 2018
We’ve recently looked at Azure AKS (Kubernetes Cluster Managed Services). We’ve looked at how to create a Kubernetes Cluster with 3 lines of Azure CLI.

With this we are able to interact with the cluster & deploy containers.

There is a bit of “auto magic” happening in...
Security with API: OAuth, token-based access vs key-based access
Solution · 11 Dec 2017
Let’s consider security with APIs, i.e how to securely identify the caller.

There are two authentication methods quite popular in the cloud to secure APIs:
1. Key-based access
2. OAuth, or token-based access in general
Let’s compare them.

Key-Based

By key-based we mean...
Azure Functions HTTP – Authorization Levels
Solution · 04 Dec 2017
In a past article, we looked at Serverless compute in Azure in general and Azure Functions specifically.

In this article we wanted to focus on Azure Function triggered by HTTP requests and the different options we have to authenticate:
- Anonymous
- Function
- Admin
- System
- User
Those are...
Managing Azure AD Application members in Portal
Solution · 19 Apr 2017
One of Azure AD’s powerful concept is the application. It gives context to an authentication as we explained in this article.

An application can also be used as an authorization barrier since we can manage an application members. This is optional as by default, everyone in a tenant has access to its application. But if we opt in to...
Automating Azure AD
Solution · 02 Feb 2017

In the previous article, we explored how to interact (read / write) to an Azure AD tenant using Microsoft Graph API.

In the article before that, we looked at how to authenticate a user without using Azure AD web flow.

Those were motivated by a...
Using Microsoft Graph API to interact with Azure AD
Solution · 31 Jan 2017

In my last article, I showed how to authenticate on Azure AD using a user name / password without using the native web flow.

The underlying scenario was to migrate an application using an LDAP server by leveraging an Azure AD tenant.

The logical continuation of that scenario is...
Authenticating to Azure AD non-interactively
Solution · 29 Jan 2017
I want to use Azure AD as a user directory but I do not want to use its native web authentication mechanism which requires users to go via an Active Directory page to login (which can be branded and customized to look like my own).

I just...
Joining an ARM Linux VM to AAD Domain Services
Solution · 18 Jan 2017
Active Directory is one of the most popular domain controller / LDAP server around.

In Azure we have Azure Active Directory (AAD). Despite the name, AAD isn’t just a multi-tenant AD. It is built for the cloud.

Sometimes though, it is useful to have a traditional domain controller... in the cloud. Typically this is with legacy workloads built to work...
Azure Active Directory Labs Series – Multi-Factor Authentication
Solution · 28 Sep 2016
Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.

In that training there were 8 labs...
Azure Active Directory Labs Series – Graph API
Solution · 21 Sep 2016
Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.

In that training there were 8 labs...
Azure Active Directory Labs Series – AD Connect
Solution · 14 Sep 2016
Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.

In that training there were 8 labs...
Azure Active Directory Labs Series – Creating an AD Forest
Solution · 11 Sep 2016
Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.

In that training there were 8 labs...
Azure Active Directory Labs Series – Adding Claims
Solution · 07 Sep 2016
Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.

In that training there were 8...
Azure Active Directory Labs Series – Protect Web App
Solution · 04 Sep 2016
Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.

In that training...
Azure Active Directory Labs Series – Create Application
Solution · 31 Aug 2016
Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.

In that training...
Azure Active Directory Labs Series - Creating a tenant
Solution · 28 Aug 2016
Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going...
Multi-Tenant SaaS with Azure Active Directory B2B & B2C
Solution · 10 Mar 2016
Scenario: I’m creating a Software as a Service (SaaS). I’m having multiple customers & I want to manage their identity. For some of my customers the users won’t have corporate identity ; I would like to offer them to login using their social identity (e.g. Facebook) if they want to or create an account on my site otherwise. For other...
Azure Active Directory Application
Solution · 10 Mar 2016
This is a quick post to talk about what an Application is for Azure Active Directory (or Azure AD or AAD).

If you come from the on premise world and are used to Active Directory on Windows Server, the concept of an application within Active Directory probably is a bit foreign.

Think about how authentication works in the...
Using Azure Active Directory Service Principal
Solution · 04 Feb 2016
You need an Azure Active Directory (AAD) identity to run some of your services: perhaps an Azure Runbook, Azure SQL Database, etc. .

You could create a normal user in Azure Active Directory and use it. If your AAD is synchronized with an on-premise one, it will get more complicated though. You will need to create it on premise and...

Integration

Broadcasting events to multiple solutions
Solution · 25 Feb 2020

I sat down with a customer who had an interesting integration problem the other day.

They are building a system that creates data useful for the entire enterprise. They want to push that data to different solutions. They also expose APIs, but they want to support the push-model where their system pushes the data...
Accessing Azure Key Vault using Managed Service Identity Logic Apps
Solution · 13 Feb 2019

Azure Key Vault is a great service to manage secrets, keys & certificates.

It uses RBAC to control access. Like all access control system, there is a chain of access. For instance, my user account has access to the vault: this means if my account’s credentials get leaked, the access to the...
Designing Distributed Systems by Brendan Burns
Solution · 09 Jan 2019

In this article I wanted to talk about a book I just finished reading.

Although its title and scope is around distributed systems in general, Brendan Burns discussion is mostly using containers & Kubernetes.

Brendan Burns is currently (as of this writing in November 2018) at Microsoft. He is a cofounder of...
Handling recursivity in Logic Apps
Solution · 28 Sep 2018

Azure Logic Apps is a powerful integration platform.

Some workflows naturally land themselves to recursion. For instance, handling a hierarchical data structure often has a natural recursive solution.

Azure Logic Apps doesn’t allow recursion directly. It forbids a Logic App invoking itself.

In this article we are going to explore two avenues...
Service Principal for Logic App Connector
Solution · 25 Sep 2018

Azure Logic Apps is a powerful integration platform.

It integrates with different services (inside and outside Azure) using connectors.

Connectors are responsible to authenticate to the service they represent.

Some connectors will hold the credentials. This is the case, for instance, of the SQL connector.

Other connectors will by default take the...
Implementing & Automating Azure Stream Analytics Pipeline
Solution · 31 May 2018
In our last article, we set out to build a resilient architecture around streaming events.

In this article, we are going to build the solution. We are going to use an ARM template which automates deployment. We will also go through the configuration of different services.

The goal...
Taming the fire hose: Azure Stream Analytics
Solution · 22 May 2018
We came upon an interesting challenge recently.

Let’s have a stream of events pouring in. The source is unimportant. Hundreds, peeking at thousands, of events per second.

We need to update a table in Azure SQL with a summary of the events.

Let’s just assume the events are about...
Setup for populating Cosmos DB with random data using Logic Apps
Solution · 01 Nov 2017
We recently published an article about Cosmos DB Performance with Geospatial Data.

In this article, we’re going to explain how to setup the environment in order to run those performance test.

More importantly, we believe this article is interesting on its own as it shows how to use...
Invoking a Stored Procedure from a partitioned CosmosDB collection from Logic Apps
Solution · 19 Oct 2017
I struggled a little to make that work, so I thought I would share the learning in order to accelerate your future endeavour.

I was looking at a way to populate a CosmosDB quickly with random data.

Stored Procedures came to mind since they would skip client-server latency. We can call a stored procedure creating hundreds of documents with random...
Multi-Tenant SaaS with Azure Active Directory B2B & B2C
Solution · 10 Mar 2016
Scenario: I’m creating a Software as a Service (SaaS). I’m having multiple customers & I want to manage their identity. For some of my customers the users won’t have corporate identity ; I would like to offer them to login using their social identity (e.g. Facebook) if they want to or create an account on my site otherwise. For other...
Integration with Azure Service Bus
Solution · 23 Aug 2015

I’ve been consulting 1.5 years for a customer embarking a journey leveraging Microsoft Azure as an Enterprise platform, helping them rethink their application park.

Characteristic of that customer:
- Lots of Software as a Service (Saas) third parties
- Business is extremely dynamic, in terms of requirements, transitions, partnerships, restructuring,...
Azure Data Factory Editor (ADF Editor)
Solution · 25 Jun 2015
Azure Data Factory is still in preview but obviously has a committed team behind it.

When I looked at the Service when the preview was made available in last Novembre, the first thing that stroke me was the lack of editor, of designing surface. Instead, you had to configure...
Azure Key Vault
Solution · 03 Feb 2015
Has somebody been peeking on my X-mas list?

Indeed, one of the weakness of the current Azure Paas solution I pointed out last year was that on non-trivial solutions you end up with plenty of secrets (e.g. user-name / password, SAS, account keys, etc.) stored insecurely in your web.config (or similar store).

I was suggesting, as a solution, to...
Large Projects
Solution · 18 Nov 2014
There is something about large projects that you'll never find, hence never learn, in smaller projects. The complexity, both technical and in terms of people dynamics, creates an all new set of challenges.

I read the article I Survived an ERP Implementation – Top 10 Gems of Advice I Learned the Hard Way at the beginning of the...
Azure ACS fading away
Solution · 04 Mar 2014
ACS is on life support for quite a while now. It was never never fully integrated to the Azure Portal, keeping the UI it had in its Azure Labs day (circa 2010, for those who were born back then).

In an article last summer, Azure Active Directory is the future of ACS, Vittorio Bertocci announces the roadmap: the demise...
Securing Azure Messaging Service Bus access
Solution · 04 Dec 2013
I am currently working on a very exciting project involving systems integration across the Azure Messaging Service Bus. I thought I would share some of the painfully acquired knowledge nuggets with you.

About %90 of examples you'll find on Internet uses Azure Bus SDK with 'owner'. That is basically 'admin' privilege because owner...
Service Bus EAI and EDI
News · 05 Feb 2012
Back in December (2011), Microsoft released the Azure Service Bus EAI & EDI Labs. This is a sort of CTP with labs.

It’s built on top of AppFabric Service Bus, but seems to contain much more than raw messaging.

According to the MSDN documentation, it seems that a lot of BizTalk concepts were re-used. For instance, look at a...
Integration in the Cloud
Solution · 17 Dec 2011
I ran across a series of good blog posts I wanted to share with you, dear readers.

From Richard Seroter, the series is about integration patterns in the cloud, looking at cloud-to-cloud & cloud-to-on-premise scenarios.

Richard looks at three main patterns:
For each...
BizTalk beyond 2010
Solution · 20 Jan 2011
I recently blogged about the future of BizTalk and the roadmap of the product version Windows AppFabric.

An even more recent blog post shed some light on the topic.
We are planning to invest in the following main areas:
- Deep Microsoft Application Platform Alignment
BizTalk vNext: BizTalk Azure?
Solution · 04 Nov 2010

There are some rumours for the next version of BizTalk (after BizTalk 2010) to be available in the Cloud. A sort of BizTalk Azure. That’s interesting news for sure.

Some are wondering how that will fit with Windows Azure AppFabric. Well, for sure, you’ll need your AppFabric Service Bus in...

Machine Learning

Quantum Computing - Scenarios
Solution · 14 Mar 2018
We recently looked at Quantum Computing value proposition.

Quantum Computer can solve some problems by several order of magnitude. This brings today’s intractable problems to be easy to solve tomorrow.

Not all problems have this property. In this article, we’ll look at some problems that do.

This is...
Hypersphere Volume
Solution · 08 Jul 2017
In our last article we looked at how the dimension of data space impacts Machine Learning algorithms. This is often referred to as the curse of dimensionality.

At the heart of the article we discussed the fact that an hypersphere hyper-volume trends to zero as dimension increases.

Here...
Hyperspheres & the curse of dimensionality
Solution · 21 Jun 2017
$fractal-1118515_640$ I previously talked about the curse of dimensionality (more than 2 years ago) related to Machine Learning.

Here I wanted to discuss it in more depth and dive into the mathematics of it.

High dimensions might sound like Physics’ string theory where our...
Training a model to predict failures
Solution · 11 May 2016
Today a quick entry to talk about a twist on Machine Learning for the predictive maintenance problem.

The Microsoft Cortana Intelligence team wrote an interesting blog the other day: Evaluating Failure Prediction Models for Predictive Maintenance.

When you listen to all the buzz around Machine Learning, it sometimes feels as if we’ve solved all the ML problems and...
How to do Data Science
Solution · 17 Apr 2016

These days, it’s all about Data Science.

What is Data Science?

Last month Brandon Rohrer, from the Cortana Intelligence and Machine Learning Blog, came up with an excellent post.

How to do Data Science?

The post basically goes over the workflow I reproduced at the right here.

I found...
What is Statistics and why should you care?
Solution · 13 Apr 2016
Unless you graduated in art, chances are you did a course in Statistics.

Chances are you hated it.

Most people I know postponed that course until the end of their degree, didn’t understand much about it and hated it dearly.

I didn’t like it either and understood very little.

A few...
Where is the statistics in Machine Learning?
Solution · 18 Oct 2015
I often try to explain what Machine Learning is to people outside the field. I’m not always good at it but I am getting better.

One of the confusion I often get when I start to elaborate the details is the presence of statistics in Machine Learning. For people outside the field, statistics are the stuff of survey or their...
Strong AI & Existential Risks
Solution · 16 Sep 2015
There has been a recrudescence of hysterical talks about Strong Artificial Intelligence (AI) lately.

Strong AI is artificial intelligence matching and eventually going beyond the full human cognitive capacity. Weak AI, by opposition, is the replication of some facets of human cognition: face recognition, voice recognition, pattern matching, etc. .

The...
Free ebook: Azure Machine Learning
Solution · 02 Sep 2015
You’re into Machine Learning, got into Azure ML, looked at my couple of blogs about it and want to take it to the next level?

Microsoft released an eBook for that exact purpose:

Free ebook: Azure Machine Learning (Microsoft Azure Essentials)

That book is targeted at...
Azure ML – Over fitting with Neural Networks
Solution · 30 Aug 2015
In a past post, I discussed the concept of over fitting in Machine Learning. I also alluded to it in my post about Polynomial Regression.

Basically, over fitting occurs when your model performs well on training data and poorly on data it hasn’t seen.

In here I’ll give an example using Artificial Neural Networks. Those can be...
AzureML - Polynomial Regression with SQL Transformation
Solution · 05 Aug 2015
I meant to illustrate over fitting (discussed in a past blog) with AzureML. An easy way to illustrate it is to fit a bunch of sample points near perfectly and the best tool for that is Polynomial Regression.

I was surprised to see that AzureML doesn’t support Polynomial Regression natively. But… while thinking about...
Azure ML - Simple Linear Regression
Solution · 18 Jul 2015
Now that we got the basics of Machine Learning out of the way, let’s look at Azure Machine Learning (Azure ML)!

In this blog, I will assume you know how to setup your workbench.

In general, there are quite a few great resources for Azure ML:
- Machine Learning Documentation Center
- Machine Learning...
Machine Learning - An Introduction - Part 2
Solution · 12 Jul 2015
In a past blog entry I gave an overview of what Machine Learning is. I showed a simple linear regression example. The goal really was to explain to newcomer to the field what Machine Learning is, what type of problem it tries to solve and what the general approach is.

Of course, I used an extremely simple example in...
Machine Learning - An introduction - Part 1
Solution · 02 Jul 2015
As I mentioned before, I did specialize (through graduated studies) in Machine Learning only to drop the field after a few years of trial on the Marketplace. I felt the field wasn’t ready for prime industrial applications.

Years have past, the field has matured and now is an exciting time to...
Twenty Years of Machine Learning at Microsoft
Solution · 12 Jan 2015
Machine Learning is the new kid on the block.

This is of special interest to me since I specialized in that field 15 years ago to "unspecialized" three years later after being discouraged by the lack of real market of the discipline. Back then (early 2000's) ML applications were always academically inspired with little true business value delivered....
Skype Translator Infographic
Solution · 07 Jan 2015
Microsoft just published an infographic about how Skype works its automatic speech recognition and translation (also known as Startrek universal translator).

I saw a demo of that technology 2 years ago, having an English and German speakers talking to each other but I didn't think it was that close to productization.

It...

Mathematics

Quantum Computing - How does it scale?
Solution · 21 Mar 2018
$fractal-1240809_640$ We recently looked at Quantum Computing value proposition. We then looked at scenarios where Quantum Computing would have a big impact.

Quantum Computer can solve some problems by several order of magnitude. This brings today’s intractable problems to be easy to solve tomorrow.

The key reason for...
Solution SLAs in Azure
Solution · 22 Jan 2018
Let’s talk about Service Level Agreement (SLA) of your solution in Azure.

Hal Berenson wrote a great article about SLA lately. It is a great conceptual background for the present today.

Here we want to focus on how you should proceed to come up with an SLA for...
Hyperspheres & the curse of dimensionality
Solution · 21 Jun 2017
$fractal-1118515_640$ I previously talked about the curse of dimensionality (more than 2 years ago) related to Machine Learning.

Here I wanted to discuss it in more depth and dive into the mathematics of it.

High dimensions might sound like Physics’ string theory where our...
What is Statistics and why should you care?
Solution · 13 Apr 2016
Unless you graduated in art, chances are you did a course in Statistics.

Chances are you hated it.

Most people I know postponed that course until the end of their degree, didn’t understand much about it and hated it dearly.

I didn’t like it either and understood very little.

A few...
Where is the statistics in Machine Learning?
Solution · 18 Oct 2015
I often try to explain what Machine Learning is to people outside the field. I’m not always good at it but I am getting better.

One of the confusion I often get when I start to elaborate the details is the presence of statistics in Machine Learning. For people outside the field, statistics are the stuff of survey or their...

Networking

Ingress rules in different Kubernetes namespaces
Solution · 11 Feb 2020

In this article I want to show how an ingress controller in Kubernetes can be used to route traffic to workloads deployed in multiple namespaces.

The online doc for AKS deploys everything in the same namespace. Hence this article is a thin extension to the online doc.

The basic trick is to deploy...
Locking down Web App to Functions communications using subnets
Solution · 03 Feb 2020

Azure Functions are serverless Function as a Service.

Serverless compute typically offers less networking options since it does not run on dedicated compute.

Until recently, the only option to lock down access to a function was to lock down the function’s firewall to some public IPs. That isn’t always useful when the...
Azure App Service Authentication with Azure Front Door
Solution · 17 Jul 2019

Last time, we looked at Azure Front Door being used as a reverse proxy in front of Azure App Service.

We are going to increase the difficulty level today by looking at how to implement Azure AD authentication(also called easy auth) in Azure App Service behind Azure Front Door.

Although that...
Azure Front Door with App Service
Solution · 11 Jun 2019

Azure Front Door service was recently released.

Azure Front Door is an interesting service combining the capabilities of:
- Reverse Proxy (SSL Termination, URL based routing, URL rewrite & session affinity)
- Web Application Firewall (WAF)
- Accelerated Global routing
- Global Load Balancing between geo-distributed backend
- Some bits of Content Delivery Network (CDN, in the...
Multiple Service Endpoints to multiple services
Solution · 18 Apr 2019

Use case: I have a central thingy that needs to talk to a service protected by a service endpoints (e.g. storage account, Azure SQL DB, Azure Maria DB, etc.). That service is also accessible to another compute in other Virtual Networks.

Is that possible?

Just to make it a little more concrete, let’s...
Testing outbound connections in AKS-Kubenet and ACI
Solution · 26 Mar 2019

What happens if a pod in AKS initiates a connection with a private endpoint? Which private IP address does the outbound connection uses?

This is relevant for a private IP inside the same VNET, a peered VNET or an IP accessible via a VPN or Express Route.

In general, the private IP of the...
Network Policies with Calico in AKS
Solution · 27 Feb 2019

Calico network plugin is finally supported within Azure Kubernetes Services (AKS).

There is a very good tutorial on the online documentation, so we won’t give a walkthrough here.

Instead we will highlight a couple of points about Network Policies in general and in AKS.

Online references

On top of the aforementioned AKS...
Understanding multiple Ingress in AKS
Solution · 28 Nov 2018

Last time, I covered the ins and outs of Ingress on AKS.

We looked at how to install nginx ingress controller and how the controller is deployed as a load balanced service. We did some URL based routing and domain name overloading.

This was in the spirit of clarifying the magic...
Understanding simple HTTP Ingress in AKS
Solution · 21 Nov 2018

We looked at Kubernetes Ingress conceptually. We looked at different use cases: URL based routing and multiple domains.

We also looked at how ingress was implemented from an AKS perspective, i.e. where traffic was routed in nodes.

In this article, I wanted to get hands on. I figured we could start...
About Ingress in Azure Kubernetes Service (AKS)
Solution · 10 Oct 2018
I did a bit of experimentation with Kubernetes Ingress, more specifically NGINX, lately.

I found the concept of Ingress utterly confusing at first. It is actually relatively simple. So, I thought I would share this sense of simplicity.

This article is conceptual. There will be no code nor even Portal tour.

If you want to ramp up on Ingress in...
AKS with Kubenet vs Azure Networking plug-in
Solution · 06 Sep 2018
Update (22-03-2019): This configuration is now officially documented.

I’ve been diving into Kubernetes / AKS Networking lately. I thought I would share some of the insights I stumble upon.

We know AKS has two types of networking, basic & advanced, right?
- Basic provisions its own VNET and exposes only public IPs
Deploying AKS with ARM Template – Network integration
Solution · 28 Aug 2018

In a past article, we looked at how Azure Kubernetes Services (AKS) integrated with Azure Networking.

AKS is a managed Kubernetes service in Azure.

In this article, we are going to do two things:
1. Deploy an AKS cluster with Advanced Networking using an Azure ARM Template.
2. Deploy a service on the...
Kubernetes Services in Azure AKS - Network integration
Solution · 21 Aug 2018

Azure Kubernetes Services (AKS) is a managed Kubernetes service in Azure. Not only does it provides a managed cluster, it also integrates with Azure services.

In this article we’ll explain the integration between Kubernetes’ services and Azure Networking. We won’t give code sample though as we’ll keep that for a future article.

…
Using Application Gateway with VNET peering
Solution · 20 Nov 2017
As I write these lines (early November 2017), Application Gateway doesn’t integrate well across VNET peering.

More precisely, if we put the gateway in a VNET and have scale sets in another, the usual integration, where the scale set registers its VM automatically as the size of the set...
Azure Virtual Machines Anatomy
Solution · 16 Oct 2017
Virtual Machines can be pretty complex little beast. They can have multiple disks, multiple NICs in different subnets, can be exposed on the public internet either directly or through a load balancer, etc. .

In this article, we’ll look at the anatomy of a Virtual Machine (VM): what are...
Virtual Network Service Endpoint - Hello World
Solution · 04 Oct 2017
In our last post we discussed the new feature Virtual Network Service Endpoint.

In this post we’re going to show how to use that feature.

We’re going to use it on a storage account.

We won’t go through the micro steps of setting up each services but we’ll focus on the Service Endpoint configuration.

Resource Group

As usual for...
VNET Service Endpoints for Azure SQL & Storage
Solution · 02 Oct 2017
It’s finally here, it has arrived: Azure Virtual Network Service Endpoints.This was a long requested “Enterprise feature”.Let’s look at what this is and how to use it.Please note that at the time of this writing (end-of-September 2017) this feature is available only in a few region in Public Preview:<ul><li>Azure...
Azure Application Gateway Anatomy
Solution · 17 Jul 2017
Back in May, we talked about Azure Application Gateway.

In this article, we’re going to look at its anatomy, i.e. its internal component as exposed in the Azure Resource Manager (ARM) model.

A lot of Azure Resource has an internal structure. For instance, a Virtual Network has a collection of subnets.

Azure Application Gateway has a very rich internal...
URL Routing with Azure Application Gateway
Solution · 08 May 2017
Update (13-06-2017): The POC of this article is available on GitHub here.

I have a scenario perfect for a Layer-7 Load Balancer / Reverse Proxy:
- Multiple web server clusters to be routed under one URL hierarchy (one domain name)
- Redirect HTTP traffic to the same URL on HTTPS
- Have reverse proxy...
Joining an ARM Linux VM to AAD Domain Services
Solution · 18 Jan 2017
Active Directory is one of the most popular domain controller / LDAP server around.

In Azure we have Azure Active Directory (AAD). Despite the name, AAD isn’t just a multi-tenant AD. It is built for the cloud.

Sometimes though, it is useful to have a traditional domain controller... in the cloud. Typically this is with legacy workloads built to work...
Troubleshooting NSGs using Diagnostic Logs
Solution · 09 Jan 2017
I’ve wrote about how to use Network Security Group (NSG) before.

Chances are, once you get a complicated enough set of rules in a NSG, you’ll find yourself with NSGs that do not do what you think they should do.

Troubleshooting NSGs isn’t trivial.

I’ll try to give some guidance here but...
Virtual Machine with 2 NICs
Solution · 23 Oct 2016
In Azure Resource Manager (ARM), Network Interface Cards (NICs) are a first class resource. You can defined them without a Virtual Machine.

UPDATE: As a reader kingly point out, NIC means Network Interface Controller, not Network Interface Card as I initially wrote. Don’t be fooled by the Azure logo ;)

Let’s...
Load Balancing VMs in Azure Resource Manager
Solution · 29 Feb 2016
Here I want to show, in details, how you would go about to expose load balanced web server VMs using Azure Resource Manager (ARM) resources.

It sounds trivial but funnily enough I didn’t find an ARM template fully doing it without bugs.

I want to explain how it works and all the moving pieces (and there are a few). I’ll...
Using Network Security Groups (NSG) to secure network access to an environment
Solution · 21 Dec 2015
Quite a few demos (including mines) ommit security for the sake of simplicity. One area where you can secure your applications in Azure is in terms of Networking.

Network Security Groups act as a firewall in the cloud.

In this post, I’ll show you how to create a virtual network with 3 subnets: front-end, middle & back-end. We’ll then secure...

NoSQL

Monitoring query performance in Cosmos DB
Solution · 04 Mar 2020

A common question with Cosmos DB is: how many Request Units (RUs) should I be using?

Although it’s easy to quickly guess what you need to start, it’s not so trivial to determine exactly what is needed.

My general advice (with Cosmos DB & other DB as a service) is to provision something...
Creating a Backup Solution for Cosmos DB using change feed
Solution · 13 Mar 2019

Cosmos DB is Azure native NoSQL database. It has awesome capabilities such as global-distribution, exceptionally high availability, throughput scalability, and much, much more.

As with most NoSQL horizontally scalable databases it doesn’t have the same backup capabilities as mainstream RDBMS systems.

Cosmos DB has an automated backup...
Cosmos DB Stored Procedures - handling continuation
Solution · 27 Jun 2018
I’ve recently did some work involving Stored Procedures in Cosmos DB.

There are a few techniques to learn when our stored procedures handle large data sets. It is all about continuation.

In this article, we’ll look at how to build a robust and scalable stored procedure.

We’ll start with...
Cosmos DB Configuration Management
Solution · 20 Jun 2018
I often found the need to automate Cosmos DB deployments.

I like to have users run an ARM template and be ready to go. Automation is key component to repeatability. It enables automated testing and increase agility in many other ways.

ARM Templates don’t go beyond a database account.

…
Disaster Recovery with VM Scale Sets & Geo-Replicated DBs
Solution · 09 Nov 2017
Last year we posted an article about different options available in Azure to implement a disaster recovery strategy.

We strongly suggest to review that article as it gives good insights about what a disaster recovery strategy is within an already resilient Cloud Environment but also clear out a...
A few CosmosDB query limitations
Solution · 06 Nov 2017
I’ve been working quite a bit with Cosmos DB since it was release in Private Preview (see my first article about it back in September 2014 as a battle scar).

It has come a long way.

From DocumentDB, the NoSQL document-oriented DB to Cosmos DB, the Globally distributed...
Setup for populating Cosmos DB with random data using Logic Apps
Solution · 01 Nov 2017
We recently published an article about Cosmos DB Performance with Geospatial Data.

In this article, we’re going to explain how to setup the environment in order to run those performance test.

More importantly, we believe this article is interesting on its own as it shows how to use...
How to Create a Logic App Connector in an ARM Template
Solution · 28 Oct 2017
I wanted to automate the performance test setup we discussed in the last article. This includes a Logic App calling into a Cosmos DB stored procedure.

Logic App uses connectors when connecting to other services, either within or outside of Azure. Now how do we...
Cosmos DB Performance with Geospatial Data
Solution · 25 Oct 2017
Time for some performance tests on Cosmos DB around Geospatial coordinates!

Let’s hurt the service and see where it shines, where it has a hard time and how scaling it (i.e. increasing Request Units or RUs) helps.

We’ll publish an how-to create the same setup in a future article...
Invoking a Stored Procedure from a partitioned CosmosDB collection from Logic Apps
Solution · 19 Oct 2017
I struggled a little to make that work, so I thought I would share the learning in order to accelerate your future endeavour.

I was looking at a way to populate a CosmosDB quickly with random data.

Stored Procedures came to mind since they would skip client-server latency. We can call a stored procedure creating hundreds of documents with random...
Hacking: changing Cosmos DB Portal experience from Graph to SQL
Solution · 10 Sep 2017
In the last article, we looked at how we could access a graph using the SQL (aka DocumentDB) API.

Here we’ll explore how we can switch the Portal experience from one to the other.

Portal Experience

The Portal Experience refers to the way the portal lets us interact with Cosmos DB Data. It’s basically the Data Explorer experience.

Here...
Hacking: accessing a graph in Cosmos DB with SQL / DocumentDB API
Solution · 05 Sep 2017
Azure Cosmos DB is Microsoft’s globally distributed multi-model database service.

At this point in time (August 2017) there are four supported models: DocumentDB (also named SQL because the query language is similar to T-SQL), MongoDB, Tabular & Gremlin.

We’ve seen how to use Cosmos DB with...
Cosmos DB Async Querying & Streaming
Solution · 31 Aug 2017
I wrote an article back in January 2015 about async querying Azure DocumentDB using the .NET SDK.

The service was still in preview back then.

Since then DocumentDB has been superseded by Azure Cosmos DB and the SDK has changed a bit so I thought I would...
Cosmos DB & Graph with Gremlin - Getting Started
Solution · 28 Aug 2017
Azure Cosmos DB is Microsoft’s globally distributed multi-model database service.

One of the paradigm it supports is Graph: Cosmos DB can be used to store and query graphs.

At the time of this writing, it supports one interface, Gremlin, which is part of the Apache...
DocumentDB protocol support for MongoDB
Solution · 20 Jun 2016
Microsoft announced, in the wake of many DocumentDB announcement, that DocumentDB would support MongoDB protocol.

What does that mean?

It means you can now swap a DocumentDB for a MongoDB and the client (e.g. your web application) will work the same.

This is huge.

It...
Azure DocumentDB Demo
Solution · 01 Dec 2015
December the 1st, 2015, I’m doing a presentation to a Montreal User Group, MS DEV MTL. Here is the script of each demo. Enjoy!

UPDATE: You can see the presentation slides here.

Account Creation & Adding Documents

For the creation of an Azure DocumentDB account, allow me to refer to myself in Creating an...
Major upgrade to Azure DocumentDB LINQ provider
Solution · 23 Sep 2015
Early Septembre 2015, Microsoft has announced a major upgrade to the LINQ Provider of the .NET SDK of DocumentDb.

I know it does appear a bit confusing since when DocumentDb was released (a year ago now), it was said that it supported SQL. Well, it supported some SQL.

Now...
Analysing Application Logs with DocumentDb
Solution · 06 Sep 2015
Azure DocumentDB is Microsoft Document-centric NoSQL offering in the cloud.

I’ve been working with it since Septembre 2014 and I wanted to share a use case I found it really good at: log analysis.

Now that takes some context.

I have been working for a customer using Microsoft Azure...
SQL Server 2016
Solution · 22 Jul 2015
Here’s a rundown of my favourite new features in SQL Server 2016, largely inspired by the SQL Server Evolution video.

Impact of Cloud-First on SQL Design

This is a really nice illustration of the consequences of Cloud-First for Microsoft products.

SQL has been basically flipped around as a product. ...
Azure DocumentDB - Performance Tips
Solution · 27 Jun 2015
Azure DocumentDB has been released for a little while now. Once you get passed the usual step of how to connect and do a few hello worlds, you will want to reach for more in-depth literature. Sooner or later, performance will be on your mind when you’ll want to take...
The limits of DocumentDB Preview Release
Solution · 11 Jan 2015
I was looking for the limits of DocumentDB Standard Tier, the only tier available during the preview release. It wasn’t all too trivial to find so here it is:

http://azure.microsoft.com/en-us/documentation/articles/documentdb-limits/

Among the limits that may constrain your solution:
- Only 3 collections per Capacity Unit (but 100 Databases, the container of collections, per account)
DocumentDB Async Querying & Streaming
Solution · 06 Jan 2015
UPDATE (31-08-2017): This article is superseded by the new article Cosmos DB Async Querying & Streaming.

Working with the .NET client SDK of Azure DocumentDB, I couldn’t find a way to query the store asynchronously.

***This post relates to the version 0.9.1-preview of Microsoft Azure DocumentDB Client Library. If you work with another major version, this might not...
Scaling a Multi-Tenant Application with Azure DocumentDB
Solution · 22 Dec 2014
Following the release of Azure DocumentDB, Microsoft NoSQL document-oriented fully managed database, in preview mode, there was initial curiosity. Now we are starting to see more focused questions: how does it scale, can it expose OData, how would I go about architecture my department app around it as the back-end, etc. .

John Macintyre, Program Manager for Azure...
Querying Collections with DocumentDB Studio
Solution · 31 Oct 2014
I released a first Release Candidate (RC) of DocumentDB Studio (release 1.0.0.1).

DocumentDB Studio is to Azure DocumentDB what SQL Management Studio is to SQL Server and SQL Azure: a one-stop shop to manage and interact with your DocumentDB.

I posted an installation guide of the application and an upgrade guide (both very simple).

…
DocumentDB Studio 1.0.0 released!
Solution · 30 Oct 2014
Yes, the first non-beta version!

Ok, what is the difference between the betas and this one? Alpha, Beta and RC are always separated by arbitrary grass lanes…

For DocumentDB Studio, I was waiting to have a Minimal Viable Product (MVP) and for me the MVP was related to be able to query document collections.

Version 1.0.0...
Managing Documents with DocumentDB Studio
Solution · 22 Oct 2014
I released a second Beta of DocumentDB Studio (release 0.2.0.1).

DocumentDB Studio is to Azure DocumentDB what SQL Management Studio is to SQL Server and SQL Azure: a one-stop shop to manage and interact with your DocumentDB.

I posted an installation guide of the application and an upgrade guide...
Upgrading DocumentDB Studio
Solution · 22 Oct 2014
See this blog post on how to install Azure DocumentDB Studio from scratch.

Now with release 0.2.0.1 just out of the door, you might want to simply upgrade.

Upgrading is basically the same thing as installing, you just want to keep the file Studio.xml around.

That file keeps your account credentials and is compatible from release...
In Azure DocumentDB, DocumentClient.CreateDocumentQuery doesn’t exists!
Solution · 19 Oct 2014
This is a quick help for you out there who are developing against Azure DocumentDB in .NET.

Maybe you read the Getting Started guide or you found that line of code somewhere else.

var families = client.CreateDocumentQuery(documentCollection.DocumentsLink, "SELECT * " + <span style="font-family:Times...
Managing Databases and Collections with DocumentDB Studio
Solution · 17 Oct 2014
I released the first Beta version of DocumentDB Studio.

DocumentDB Studio is to Azure DocumentDB what SQL Management Studio is to SQL Server and SQL Azure: a one-stop shop to manage and interact with your DocumentDB.

I posted an installation guide of the application (very simple)....
Installing DocumentDB Studio
Solution · 16 Oct 2014
As I announced yesterday, I released the first Beta version of DocumentDB Studio.

DocumentDB Studio is to Azure DocumentDB what SQL Management Studio is to SQL Server and SQL Azure: a one-stop shop to manage and interact with your DocumentDB.

In this blog post I'll simply walk you through the installation process.

…
Creating an Azure DocumentDB account
Solution · 16 Oct 2014
In this post I will guide you on how to create an Azure DocumentDB account.

If you remember the Azure DocumentDB resource model, the entire resource model is rooted at the Database account:

So, step by step:
1. Go to the preview portal: https://portal.azure.com/
2. Click the NEW button at the bottom...
DocumentDB Studio first (beta) release
Solution · 15 Oct 2014
I finally done a first release of my latest project, DocumentDB Studio!

DocumentDB Studio is to Azure DocumentDB what SQL Management Studio is to SQL Server and SQL Azure: a one-stop shop to manage and interact with your DocumentDB.

The first release is quite minimalist and only let you:
- Connect to...
Profile of MSN Health and Fitness on Azure DocumentDB
Solution · 13 Oct 2014
We already know that Azure DocumentDB was used in the back-end of OneNote. Last week Microsoft released another nice case study: the user profiles for MSN portal.

The requirements for the solution were:
1. Scale requirements to support +425M unique MSN users with +100M direct authenticated users. Initial capacity requirements for 20TB of document storage.
2. Under 15ms...
NoSQL implementation concepts
Solution · 07 Oct 2014
While familiarizing ourselves with Microsoft new DocumentDB, a solid foundation on NoSQL doesn't hurt.

A few years ago, I saw a couple of great presentations on Channel9 (from a TechED if I am not mistaken) about Azure Storage. The presenters went into how partitioning works, how requests are routed, how consistency is ensure,...
Digest: DocumentDB Resource Model and Concepts
Solution · 18 Sep 2014
Azure DocumentDB has release a few weeks ago and with it an early, in small quantity, of good quality documentation.

One of those article is DocumentDB Resource Model and Concepts. That article goes through the different concepts of the inner model of DocumentDB.

That article sheds some light on the product but also reveal the...
Azure DocumentDB: first use cases
Solution · 08 Sep 2014
A few weeks ago Microsoft released (in preview mode) its new NoSQL Database: DocumentDB.

Not Only SQL (NoSQL) databases are typically segmented in the following categories: Key-Value (e.g. Azure Table Storage, Redis), Column (e.g. HBase, Cassandra), Document (e.g. CouchDB, MongoDB) & Graph. By its name but mostly by its feature set, DocumentDB falls in the document category.

…

Office365

Unit Testing SharePoint
Solution · 30 Nov 2012
Doing automated unit tests in SharePoint isn’t easy.

As with all libraries that haven’t been designed with unit testing in mind, SharePoint object model doesn’t expose its dependencies: it connects to a Content Database given the context creating it and there are no ways to redirect it to some stub implementations.

That is unless you can override method invocations. This...
Creating numbered headings or outline numbering
Gadget · 18 Mar 2011
Far from fancy technological questions today…

How do you configure Microsoft Word to assign numbers to your section headers in a hierarchical way, e.g.:
- Header 1 (1)
- Header 1 (2)
- Header 2 (2.1)
- Header 2 (2.2)
- Header 3 (2.2.1)
etc.?

Well, maybe you’re in my position and you always had documents...
Office 365 Beta Service Descriptions
Solution · 20 Dec 2010
Microsoft has made available the descriptions of Office 365 Beta Services:

http://www.microsoft.com/downloads/en/details.aspx?FamilyID=6c6ecc6c-64f5-490a-bca3-8835c9a4a2ea

I went through the Exchange Online Services & SharePoint Services. I noted the following features for Office 365 in general:
- Secure Access (SSL 128-bits)
- Security Audits:
  - ISO 27001
  - SAS70 Type I & II audits...
The road to Office 365
Solution · 03 Dec 2010
Mary-Jo Foley has written a nice series of articles on her blog:
In the series she traces back the origin of Office 365 (BPOS) into what it is today...
Windows 8: Desktop as a Service?
Solution · 16 Nov 2010
In the wild country of rumours about Windows 8, there’s a new entry: Desktop as a Service (thanks to Mary-Jo Foley for the heads-up). Some slides have indeed leaked from the London Microsoft architectural summit in April 2010 showing Microsoft’s vision of the next step for Windows virtualization.

The virtualization of the...
Office Web App to power Facebook emails
News · 16 Nov 2010
The new Facebook mail service will use Microsoft Office Web App in order to view Microsoft Office documents.

This follows the news of Facebook using Bing to search the social network.

It’s interesting to see Microsoft positioning itself in the social networking space and the search space. It recently abandoned the idea of powering Livre Blog by...
BPOS is dead, long live Office 365
Solution · 19 Oct 2010
Starting in early 2010, what was previously known as BPOS version 2 or Union will be available as Office 365. The name should reflect the availability of the platform (which suffered some reputation step back this summer). Today the beta is released, available to a 1000+ customers worldwide.

Among...
BPOS SharePoint 2010
Solution · 15 Jul 2010
I finally read an official note from the BPOS team about a bunch of features I was looking for in BPOS, especially for SharePoint. Mainly:
- Federated Identity!
- SharePoint 2010 Online
- Extranet access for SharePoint online
There are no date mentioned, but BPOS usually deploys new version quarterly.

As I mentioned in...
Microsoft Online (BPOS)
Solution · 09 Jul 2010
I’ve been looking at the BPOS offering for my company. Since the documentation is quite spread-out on the web, I though a blog post about my findings would be useful.

Terminology

First thing, let’s clear out the terminology. As it is sometimes the case with Microsoft products, it’s not a simple thing.

Microsoft Online is MS offering for Cloud Services. ...

Operation

Change Management in a Kusto Database
Solution · 26 Aug 2020

Let’s say you’ve setup an Azure Data Explorer cluster. You’ve also setup real time ingestion pipeline. If you haven’t, take a look at the Real Time Lab I’ve put together, where you’re going to build pipelines and query data in real time.

Now this has been running for weeks, maybe months.

You...
My ultimate ingestion process for historical data at scale with Kusto
Solution · 10 Jun 2020

In my last article, we discussed different architecture aspects of large historical data ingestion.

In this article, I want to be more prescriptive and share an approach that works well for me.

Is that the ultimate process? Of course not, that is clickbait. As we discussed at length in the previous article, different...
Ingesting historical data at scale with Kusto
Solution · 03 Jun 2020

There are many ways to ingest data in Kusto.

There is batching vs streaming, queued vs command, plugins, SDK, etc. . There is also a plethora of tools / techniques to achieve this, e.g. Azure Data Factory, LightIngest, .ingest into, etc. .

In this article I want to look...
Starting / Stopping Kusto cluster with Logic App
Solution · 22 Apr 2020

In past articles, we looked at how to archive Azure Monitor data using Kusto (ADX) & how to automate that process using Azure Logic Apps.

If the sole purpose of that specific Kusto Cluster is to archive Azure Monitor Data (as it is in my case for my blog), there is no...
Automating archiving Azure Monitor Data with Kusto
Solution · 15 Apr 2020

In our last article, we laid down a method on how to archive Azure Monitor Data using Kusto (Azure Data Explorer). This allows us to later analyse that data on a much longer period than the Azure Monitor retention period.

In this article, we will automate that archiving...
Archiving Azure Monitor Data with Kusto
Solution · 08 Apr 2020
Around the turn of the year, I moved my blog to GitHub pages. GitHub pages does only the static content serving part. I surrounded it with different Azure services to have a complete blogging solution.

One of those services is Azure Application Insights for web analytics.

App Insights keeps the data...

Operations

Monitoring query performance in Cosmos DB
Solution · 04 Mar 2020

A common question with Cosmos DB is: how many Request Units (RUs) should I be using?

Although it’s easy to quickly guess what you need to start, it’s not so trivial to determine exactly what is needed.

My general advice (with Cosmos DB & other DB as a service) is to provision something...
Tracing request in Azure API Management
Solution · 13 Nov 2019

Azure API Management (API-M) is an API Gateway solution.

It is quite easy to create an API in API-M, connect it to a back-end API and test it in the portal.

What happen when you test it from another client platform (e.g. Postman) and it fails? You have no visibility into what...
Alerts on Azure Function failures
Solution · 01 Nov 2019

So, you have a few functions running. Maybe some of those functions are important and you would like to be alerted when they fail.

In this article I’ll cover that exact scenario. We will do that in the following steps:
- Deploy a recurrent function which fails %50 of the time (by design)
- Define...
Monitoring metrics in AKS
Solution · 09 May 2019

AKS has a nice integration with Azure monitor. Out of the box there are a couple of dashboards for common metrics.

What if you need to go beyond those metrics?

This is what we’re going to do in this article. I’m going to show how to get the CPU usage per container. Along...
Creating a Backup Solution for Cosmos DB using change feed
Solution · 13 Mar 2019

Cosmos DB is Azure native NoSQL database. It has awesome capabilities such as global-distribution, exceptionally high availability, throughput scalability, and much, much more.

As with most NoSQL horizontally scalable databases it doesn’t have the same backup capabilities as mainstream RDBMS systems.

Cosmos DB has an automated backup...
Custom Logs on AKS & Azure Monitor
Solution · 30 Jan 2019

Let’s look at a concrete problem:
- I have containers deployed in AKS
- Those container log into custom files
- I want to analyse those logs using Azure Monitor (Log Analytics)
We’ll look at how to do that.

We leverage Azure Monitor for containers.

Custom logs in Log Analytics also is...

PowerShell

Finding a VM Image Reference, Publisher & SKU
Solution · 10 Jan 2018
Automation is great.

I love to script solutions in Azure.

But most of the time, that automation started with me fumbling and trialing different approaches in the portal. Once I got something around what I wanted I’ll automate.

I suspect you do the same thing.

This is why today we’re going...
Moving from Standard to Premium disks and back
Solution · 27 Sep 2017
Azure Managed Disks (introduced in February 2017) simplified the way Virtual Machine disks are managed in Azure.

A little known advantage of that resource is that it exposes its storage type, i.e. Standard vs Premium, as a simple property that can easily be changed.

Why would we do that? Typically we’ll move from standard to premium storage to improve...
How to know where a Service is Available in Azure
Solution · 14 Sep 2017
Azure has a Global Footprint of 40 regions at the time of this writing (mid-September 2017).

Not all services are available in every regions. Most aren’t in fact. Only foundational services (e.g. storage) are available everywhere.

In order to know where a service is available, we...
Azure SQL Elastic Pool – Moving databases across pools using PowerShell
Solution · 08 Jan 2017

I’ve written a bit about Azure SQL Elastic Pool lately: an overview, about ARM template and about database size.

One of the many great features of Azure SQL Elastic Pool is that like Azure SQL Database (standalone), we can change the eDTU capacity of the pool...
Finding ARM template ApiVersion
Solution · 19 Feb 2016
Writing an Azure ARM template for a Resource Group is getting easier every day but it remains a sport for the initiated.

Here I want to give a tip about something I often find hard: how to get the API version of a resource in an ARM template?

As everything hardcore in the platform,...
Listing Resources under Resource Group with Azure PowerShell
Solution · 21 Jan 2016
Simple task: I want to list resources under a single ResourceGroup within one of the subscription.

This is an excuse to show how to login on different subscription and play a bit with the PowerShell Azure SDK.

First thing, start PowerShell Integrated Script Environment (ISE) with the Azure SDK loaded.

Login

Login to your accounts…
```
<span...
```
Move Azure Resources between Resource Groups using Powershell
Solution · 28 Oct 2015
Ouf… I’ve been using Azure for quite a while in the old (current actually) portal. Now I look into my resources in the new (preview) portal and… what a mess of a resource group mosaic!

Unfortunately, at the time of this writing, you can’t move resources from a Resource Group to another via the portal…

If you’ve been there,...

Quantum

Quantum Computing - How does it scale?
Solution · 21 Mar 2018
$fractal-1240809_640$ We recently looked at Quantum Computing value proposition. We then looked at scenarios where Quantum Computing would have a big impact.

Quantum Computer can solve some problems by several order of magnitude. This brings today’s intractable problems to be easy to solve tomorrow.

The key reason for...
Quantum Computing - Scenarios
Solution · 14 Mar 2018
We recently looked at Quantum Computing value proposition.

Quantum Computer can solve some problems by several order of magnitude. This brings today’s intractable problems to be easy to solve tomorrow.

Not all problems have this property. In this article, we’ll look at some problems that do.

This is...
Quantum Computing - Value Proposition
Solution · 07 Mar 2018
$fractal-1280081_640$

Quantum Computing has gone from being an academic curiosity to a big bet in the last decade. Microsoft, Google, IBM and plenty of others are investing into that technology.

There still isn’t a large scale Quantum Computer commercially availably, but many think it’s around...

Security

Recursive ADLS Access Control
Solution · 29 Jul 2020

Last time we showed how to use Logic App to invoke ADLS REST APIs (list blobs specifically) using AAD identities (in this case, Azure’s Managed Service Identities, MSI).

In the past, we discussed access control in Azure Data Lake Storage (ADLS). Those intricacies are useful when accessing ADLS using Azure AD authentication.

…
Azure Data Lake Storage Logic App with Managed Identities
Solution · 22 Jul 2020

Last time we discussed some gotcha with Azure Data Lake Storage (ADLS) and access control. Those intricacies are useful when accessing ADLS using Azure AD authentication.

Unfortunately, Azure AD authentication is a little more than one year old, so a lot of tools are still using the good old storage account access keys....
Access Control in Azure Data Lake Storage
Solution · 16 Jul 2020

About a year ago I did an article about Azure Data Lake Storage (ADLS) gen 2 and how to use its REST API.

ADLS gen 2 unlocked a bunch of scenarios. It is present in every region, it enables analytics operations (e.g. renaming folders), it supports HDFS protocol, it scales, etc. .

In...
Impersonating user in ADLS with Kusto
Solution · 27 Feb 2020

We discussed Azure Data Explorer (ADX) and its query language Kusto in a past article.

In this article I wanted to show how to access an ADLS gen 2 using user impersonation, i.e. using the identity of the user running the queries to access the storage.

It would be all trivial and boring...
Locking down Web App to Functions communications using subnets
Solution · 03 Feb 2020

Azure Functions are serverless Function as a Service.

Serverless compute typically offers less networking options since it does not run on dedicated compute.

Until recently, the only option to lock down access to a function was to lock down the function’s firewall to some public IPs. That isn’t always useful when the...
Accessing Azure Key Vault from within Azure API Management
Solution · 19 Nov 2019

Today we look at a common although slightly advanced scenario with API Management: accessing Azure Key Vault from Azure API Management.

In an Enterprise, API Management service are often shared between teams. This means a lot of people might open it in the Portal and look at it. It also means that putting...
API Management - OAuth and private back-ends
Solution · 31 Jul 2019

Azure API Management is a fully managed API Gateway service.

In my last article we looked at the anatomy of the service. Today I wanted to demonstrate how to use OAuth with JWT token to protect an API Front End. I also wanted to show how we can access backend APIs...
Azure App Service Authentication with Azure Front Door
Solution · 17 Jul 2019

Last time, we looked at Azure Front Door being used as a reverse proxy in front of Azure App Service.

We are going to increase the difficulty level today by looking at how to implement Azure AD authentication(also called easy auth) in Azure App Service behind Azure Front Door.

Although that...
Azure Front Door with App Service
Solution · 11 Jun 2019

Azure Front Door service was recently released.

Azure Front Door is an interesting service combining the capabilities of:
- Reverse Proxy (SSL Termination, URL based routing, URL rewrite & session affinity)
- Web Application Firewall (WAF)
- Accelerated Global routing
- Global Load Balancing between geo-distributed backend
- Some bits of Content Delivery Network (CDN, in the...
Authenticating an Azure service principal
Solution · 22 May 2019

When it comes to using Service Principal in Azure, I always advise using Managed System Identity (MSI).

MSI is simpler and safer. MSI handles certificate rotations. We never see the certificate. Remember this: the safest secret is the secret you never see.

But in cases we can’t use MSI, we are going...
Multiple Service Endpoints to multiple services
Solution · 18 Apr 2019

Use case: I have a central thingy that needs to talk to a service protected by a service endpoints (e.g. storage account, Azure SQL DB, Azure Maria DB, etc.). That service is also accessible to another compute in other Virtual Networks.

Is that possible?

Just to make it a little more concrete, let’s...
Testing outbound connections in AKS-Kubenet and ACI
Solution · 26 Mar 2019

What happens if a pod in AKS initiates a connection with a private endpoint? Which private IP address does the outbound connection uses?

This is relevant for a private IP inside the same VNET, a peered VNET or an IP accessible via a VPN or Express Route.

In general, the private IP of the...
Flex Volume in AKS
Solution · 21 Feb 2019

I wanted to start looking at a few modules helping integrate AKS with the rest of Azure.

In a past article, we looked at Pod Identity.

This time around, we will look at Flex Volume.

Flex Volume is an integration between AKS and Key Vault. It allows keys, secrets & certificates (the...
Azure AD Pod Identity in AKS
Solution · 19 Feb 2019

I wanted to start looking at a few modules helping integrate AKS with the rest of Azure.

A big integration point is identity. For many reasons, we’ll want our pods to use service principal identities:
1. Access an Azure service supporting AAD-integration
Accessing Azure Key Vault using Managed Service Identity Logic Apps
Solution · 13 Feb 2019

Azure Key Vault is a great service to manage secrets, keys & certificates.

It uses RBAC to control access. Like all access control system, there is a chain of access. For instance, my user account has access to the vault: this means if my account’s credentials get leaked, the access to the...
Service Principal for Logic App Connector
Solution · 25 Sep 2018

Azure Logic Apps is a powerful integration platform.

It integrates with different services (inside and outside Azure) using connectors.

Connectors are responsible to authenticate to the service they represent.

Some connectors will hold the credentials. This is the case, for instance, of the SQL connector.

Other connectors will by default take the...
Creating a Service Principal with Azure CLI
Solution · 23 Aug 2018

Service Principals are a bit of a weird beast. They are Azure Active Directory applicationswith kind of an extra bit. That bit says they can actually login by themselves. Hence the name principal. But being an application is kind of weird.

We covered Service Principals in the past. We covered how to...
RBAC and role assignment using ARM Templates
Solution · 15 Aug 2018
Azure supports Role Based Access Control (RBAC) as an access control paradigm.

It allows to map a user (or a group of users) to a role within a given scope (resource, resource group, subscription or management group).

For instance, we could map my user identity to a Virtual...
Understanding identities in Azure AKS / Kubernetes
Solution · 10 May 2018
We’ve recently looked at Azure AKS (Kubernetes Cluster Managed Services). We’ve looked at how to create a Kubernetes Cluster with 3 lines of Azure CLI.

With this we are able to interact with the cluster & deploy containers.

There is a bit of “auto magic” happening in...
Security with API: OAuth, token-based access vs key-based access
Solution · 11 Dec 2017
Let’s consider security with APIs, i.e how to securely identify the caller.

There are two authentication methods quite popular in the cloud to secure APIs:
1. Key-based access
2. OAuth, or token-based access in general
Let’s compare them.

Key-Based

By key-based we mean...
Azure Functions HTTP – Authorization Levels
Solution · 04 Dec 2017
In a past article, we looked at Serverless compute in Azure in general and Azure Functions specifically.

In this article we wanted to focus on Azure Function triggered by HTTP requests and the different options we have to authenticate:
- Anonymous
- Function
- Admin
- System
- User
Those are...
Virtual Network Service Endpoint - Hello World
Solution · 04 Oct 2017
In our last post we discussed the new feature Virtual Network Service Endpoint.

In this post we’re going to show how to use that feature.

We’re going to use it on a storage account.

We won’t go through the micro steps of setting up each services but we’ll focus on the Service Endpoint configuration.

Resource Group

As usual for...
VNET Service Endpoints for Azure SQL & Storage
Solution · 02 Oct 2017
It’s finally here, it has arrived: Azure Virtual Network Service Endpoints.This was a long requested “Enterprise feature”.Let’s look at what this is and how to use it.Please note that at the time of this writing (end-of-September 2017) this feature is available only in a few region in Public Preview:<ul><li>Azure...
Automating Role Assignment in Subscriptions & Resource Groups
Solution · 05 May 2017
Azure supports a Role Based Access Control (RBAC) system. This system links identity (users & groups) to roles.

RBAC is enforced at the REST API access level, which is the fundamental access in Azure: it can’t be bypassed.

In this article, we’ll look at how we can automate the role...
Managing Azure AD Application members in Portal
Solution · 19 Apr 2017
One of Azure AD’s powerful concept is the application. It gives context to an authentication as we explained in this article.

An application can also be used as an authorization barrier since we can manage an application members. This is optional as by default, everyone in a tenant has access to its application. But if we opt in to...
Automating Azure AD
Solution · 02 Feb 2017

In the previous article, we explored how to interact (read / write) to an Azure AD tenant using Microsoft Graph API.

In the article before that, we looked at how to authenticate a user without using Azure AD web flow.

Those were motivated by a...
Using Microsoft Graph API to interact with Azure AD
Solution · 31 Jan 2017

In my last article, I showed how to authenticate on Azure AD using a user name / password without using the native web flow.

The underlying scenario was to migrate an application using an LDAP server by leveraging an Azure AD tenant.

The logical continuation of that scenario is...
Authenticating to Azure AD non-interactively
Solution · 29 Jan 2017
I want to use Azure AD as a user directory but I do not want to use its native web authentication mechanism which requires users to go via an Active Directory page to login (which can be branded and customized to look like my own).

I just...
Primer on Azure Monitor
Solution · 27 Nov 2016

Azure Monitor is the latest evolution of a set of technologies allowing Azure resources monitoring.

I’ve written about going the extra mile to be able to analyze logs in the past.

The thing is that once our stuff is in production with tons of users hitting...
Securing REST API using Azure Active Directory
Solution · 11 Mar 2016
Scenario: you have a web & mobile front-end, both using a REST API as a back-end. You want to secure that back-end with authentication / authorization. How do you do that in Azure?

There are obviously a bunch of ways to do that. In this post, I’ll discuss the recommended...
Multi-Tenant SaaS with Azure Active Directory B2B & B2C
Solution · 10 Mar 2016
Scenario: I’m creating a Software as a Service (SaaS). I’m having multiple customers & I want to manage their identity. For some of my customers the users won’t have corporate identity ; I would like to offer them to login using their social identity (e.g. Facebook) if they want to or create an account on my site otherwise. For other...
Azure Active Directory Application
Solution · 10 Mar 2016
This is a quick post to talk about what an Application is for Azure Active Directory (or Azure AD or AAD).

If you come from the on premise world and are used to Active Directory on Windows Server, the concept of an application within Active Directory probably is a bit foreign.

Think about how authentication works in the...
Row Level Security in 3-tiers applications
Solution · 09 Feb 2016
Row-Level Security is a great new feature in SQL. It is already available in Azure SQL Database and will be available in SQL Server 2016.

This is Cloud-First for you.

Row-Level Security allows you to forbid a user to see and/or access data they shouldn’t be able to see. Traditionally this is implemented at the application level, for...
Azure Key Vault & SQL Server Connector Update
Solution · 07 Oct 2015
Azure Key Vault is alive and well!

The Azure service allowing you to store keys and secrets in a secured container has been released at the end of summer and it continues to improve.

The SQL Server Connector is a component that can be installed on SQL Server...
Azure Key Vault is now generally available
Solution · 09 Jul 2015
Azure Key Vault is now generally available!

What does it change? Well, it gives us way more stability in the service in terms of API surface but also, it backs the entire service with an SLA!

You can read some of my past articles about Azure Key Vault:
- Introduction
- Step by step
Azure Key Vault - Pricing
Solution · 20 Jun 2015
Azure Key Vault is an Azure packaged service allowing you to encrypt keys and small secrets (e.g. passwords, SAS) and manage them in a secure fashion. Azure Key Vault actually allows you to store cryptographic keys and do operations with them (e.g. encrypt data) without revealing the key, which is pretty cool. Check it out.

Now a common...
Azure Key Vault – Step by Step
Solution · 13 Jun 2015
Azure Key Vault is an Azure packaged service allowing you to encrypt keys and small secrets (e.g. passwords, SAS) and manage them in a secure fashion. Azure Key Vault actually allows you to store cryptographic keys and do operations with them (e.g. encrypt data) without revealing the key, which is pretty cool. Check it out.

A typical problem...
Azure Key Vault
Solution · 03 Feb 2015
Has somebody been peeking on my X-mas list?

Indeed, one of the weakness of the current Azure Paas solution I pointed out last year was that on non-trivial solutions you end up with plenty of secrets (e.g. user-name / password, SAS, account keys, etc.) stored insecurely in your web.config (or similar store).

I was suggesting, as a solution, to...
Surveying the Web API Security Landscape
Solution · 10 Jan 2015
Ronnie Mitra published a nice review of the current security protocols & standards available to you to protect your API.

Ronnie gives a good overview of what used to be available, in SOAP WS-*, what is currently available (HTTPS, OAuth and the like) and what is brewing for the future (Oz & Macaroons).
How to improve Azure: Granularity of access
Solution · 03 May 2014
In this blog series I explore some of the shortcomings of the Microsoft Azure platform (as of this date, April 2014) and discuss ways it could be improved. This isn’t a rant against the platform: I’ve been using and promoting the platform for more than four (4) years now and I’m very passionate about it. Here I am pointing...
How to improve Azure: Security Models
Solution · 28 Apr 2014
In this blog series I explore some of the shortcomings of the Microsoft Azure platform (as of this date, April 2014) and discuss ways it could be improved. This isn’t a rant against the platform: I’ve been using and promoting the platform for more than four (4) years now and I’m very passionate about it. Here I am pointing...
How to improve Azure: Can you keep a secret?
Solution · 07 Mar 2014
In this blog series I explore some of the shortcomings of the Windows Azure platform (as of this date, March 2014) and discuss ways it could be improved. This isn’t a rant against the platform: I’ve been using and promoting the platform for more than four (4) years now and I’m very passionate about it. Here...
Securing Azure Messaging Service Bus access
Solution · 04 Dec 2013
I am currently working on a very exciting project involving systems integration across the Azure Messaging Service Bus. I thought I would share some of the painfully acquired knowledge nuggets with you.

About %90 of examples you'll find on Internet uses Azure Bus SDK with 'owner'. That is basically 'admin' privilege because owner...
New Microsoft Services Single Sign-In UX
Solution · 07 Feb 2013
Microsoft is going to deploy a new simplified user experience to login in its different services.

You can opt-in to this new UX by going here. You will have to opt-in every week though.
Advertisement tracking in iOS 6 vs no-tracking in IE 10
Gadget · 25 Oct 2012
As the Business Insider reported, Apple has quietly introduced a new technology called IDFA (Identification For Advisers) in iOS 6.

This technology is similar to cookies in browser and advertisers to know which iPhone user has looked at which ads on which web site. It is anonymous in the sense that no other information beyond your ID is...
Departmental Application Migration to Azure Blog Series
Solution · 16 Dec 2011
Back in summer 2010 I did a proof of concept around Windows Azure & ADFS. The challenge was:

How can we deploy a departmental application in the cloud and have employees connecting on it using their corporate account?

Basically, how to project a corporate account in the cloud?

This was before Azure AppFabric Access Control. The solution had...
Everything you ever wanted to know about SQL Azure Connection Security
Solution · 27 Jun 2011
See this TechNet article for a complete description of all the security topics:

http://social.technet.microsoft.com/wiki/contents/articles/sql-azure-connection-security.aspx
January 2011 Releases of the Identity Training Kit & the MSDN Online Identity Training Course
Solution · 24 Jan 2011
You can read more details on Vittorio Bertocci’s excellent blog.

You can find the training kit here and the courses here.

The training kit is an update from last June’s and now contain elements about ADFS, Windows Azure AppFabric Access Control v 1.0 (Septembre 2010) and Access Control Lab (Decembre 2010).

The courses seem to be migrated...
Single Sign-On from Active Directory to a Windows Azure Application Whitepaper
Solution · 12 Jan 2011
A new whitepaper on how to connect to Windows Azure hosted applications using Active Directory has been published in December 2010.

The two major building blocks are WIF & ADFS.

I did cover this topic in a series of blog last summer. Now you have a nicely packaged whitepaper explaining it!

Also, Microsoft has published a series...
ADFS 2.0 Step by Step and How To guides
Solution · 23 Nov 2010
Microsoft released a series of document guiding on how to use ADFS 2.0:

http://technet.microsoft.com/en-us/library/dd727938(WS.10).aspx

However useful my blog series on how to setup ADFS 2.0 to be used with Windows Azure was, I’m sure those guides will also be useful ;)
Departmental Application Migration to Azure – Part 4 – ADFS with Azure web app
Solution · 25 Aug 2010
This is part of a series of blogs. See the preceding blog entries:
As mentioned before on this...
ASP.NET + WIF article
Solution · 10 Aug 2010
Here’s an MSDN article about how to integrate WIF into an ASP.NET application: Passive Authentication for ASP.NET with WIF.

It doesn’t talk about ADFS it goes into the details of the configuration of WIF. That’s very interesting since the STS Wizard generates a bunch of configuration but no explanation on what is doing what.
Expended feature set in Access Control Service (AppFabric LABS)
Solution · 06 Aug 2010
Microsoft just released a new feature set on top of Access Control Service (ACS). It is currently in “incubation” in AppFabric Labs. The Azure Labs are charge-free developer previews of features. They should eventually move to “production” with charge patterns.

You can see a quick (5 minutes) demo of it on Wade Wegner blog’s. Wade is a Microsoft Technical...
Departmental Application Migration to Azure – Part 3 – ADFS with on-premise web app
Solution · 15 Jul 2010
This is part of a series of blogs. See the preceding blog entries:
- Departmental Application Migration to Azure – Part 1
- Departmental Application Migration to Azure – Part 2 – ADFS Installation
For me, authentication is the major challenge for this proof of concept. The migration of our tiny database, the web application that...

Serverless

Locking down Web App to Functions communications using subnets
Solution · 03 Feb 2020

Azure Functions are serverless Function as a Service.

Serverless compute typically offers less networking options since it does not run on dedicated compute.

Until recently, the only option to lock down access to a function was to lock down the function’s firewall to some public IPs. That isn’t always useful when the...
Alerts on Azure Function failures
Solution · 01 Nov 2019

So, you have a few functions running. Maybe some of those functions are important and you would like to be alerted when they fail.

In this article I’ll cover that exact scenario. We will do that in the following steps:
- Deploy a recurrent function which fails %50 of the time (by design)
- Define...
API Management exclusive access to Azure Function
Solution · 28 Mar 2019

[Update 05-04-2019: Erratum on the original article. Logic Apps is actually able to perform public IP filering.]

Azure API Management acts as a front door to your APIs.

Typically, we do not want users / apps to be able to access the underlying APIs directly since that would bypass the API Management policies,...
Testing outbound connections in AKS-Kubenet and ACI
Solution · 26 Mar 2019

What happens if a pod in AKS initiates a connection with a private endpoint? Which private IP address does the outbound connection uses?

This is relevant for a private IP inside the same VNET, a peered VNET or an IP accessible via a VPN or Express Route.

In general, the private IP of the...
My JSON gets serialized in a string - Logic Apps
Solution · 06 Mar 2019

Azure Logic Apps is a powerful tool.

It takes care of long running tasks, retries, bunch of integrations, etc . .

It also handles JSON payloads natively. JSON flows in and out of an app.

I recently faced a bit of a problem with JSON though.

My JSON was appearing serialized within a string....
Accessing Azure Key Vault using Managed Service Identity Logic Apps
Solution · 13 Feb 2019

Azure Key Vault is a great service to manage secrets, keys & certificates.

It uses RBAC to control access. Like all access control system, there is a chain of access. For instance, my user account has access to the vault: this means if my account’s credentials get leaked, the access to the...
Using Azure DevOps REST API to start multiple releases with Azure Logic Apps
Solution · 12 Dec 2018

In our last article, we looked at how we could leverage the Azure DevOps REST API to trigger multiple releases.

This is useful when rebuilding an environment: first we release the shared infrastructure then each service on top of it.

In this article, we’ll see an implementation example using Azure Logic...
Handling recursivity in Logic Apps
Solution · 28 Sep 2018

Azure Logic Apps is a powerful integration platform.

Some workflows naturally land themselves to recursion. For instance, handling a hierarchical data structure often has a natural recursive solution.

Azure Logic Apps doesn’t allow recursion directly. It forbids a Logic App invoking itself.

In this article we are going to explore two avenues...
Service Principal for Logic App Connector
Solution · 25 Sep 2018

Azure Logic Apps is a powerful integration platform.

It integrates with different services (inside and outside Azure) using connectors.

Connectors are responsible to authenticate to the service they represent.

Some connectors will hold the credentials. This is the case, for instance, of the SQL connector.

Other connectors will by default take the...
Implementing & Automating Azure Stream Analytics Pipeline
Solution · 31 May 2018
In our last article, we set out to build a resilient architecture around streaming events.

In this article, we are going to build the solution. We are going to use an ARM template which automates deployment. We will also go through the configuration of different services.

The goal...
Taming the fire hose: Azure Stream Analytics
Solution · 22 May 2018
We came upon an interesting challenge recently.

Let’s have a stream of events pouring in. The source is unimportant. Hundreds, peeking at thousands, of events per second.

We need to update a table in Azure SQL with a summary of the events.

Let’s just assume the events are about...
How to get rid of /api in Azure Function’s route?
Solution · 14 Feb 2018
We looked at Azure Functions.

We also looked at security around Azure Function used to implement APIs.

Something people will quickly notice when implementing an Webhook / API function is that its URL or route is always prepended by /api.

For instance, if we create a webhook...
Recurrent serverless batch job with Azure Batch
Solution · 20 Dec 2017
Most solutions have recurrent batch jobs, e.g. nightly / end-of-month batch jobs.

There are many services we can leverage in Azure to run those. In this article, we are going to explore a service that has “Batch” in its name: Azure Batch.

Azure Batch is typically positioned...
Azure Functions HTTP – Authorization Levels
Solution · 04 Dec 2017
In a past article, we looked at Serverless compute in Azure in general and Azure Functions specifically.

In this article we wanted to focus on Azure Function triggered by HTTP requests and the different options we have to authenticate:
- Anonymous
- Function
- Admin
- System
- User
Those are...
Serverless compute with Azure Functions - Getting Started
Solution · 27 Nov 2017
I meant to write about Azure Functions & serverless computing for quite a while now. So here we go.

This entry is meant as a “getting started” entry with pointers to documentation and general discussion. We’ll dive deeper in future articles.

Serverless

What is serverless computing?

…
Setup for populating Cosmos DB with random data using Logic Apps
Solution · 01 Nov 2017
We recently published an article about Cosmos DB Performance with Geospatial Data.

In this article, we’re going to explain how to setup the environment in order to run those performance test.

More importantly, we believe this article is interesting on its own as it shows how to use...
How to Create a Logic App Connector in an ARM Template
Solution · 28 Oct 2017
I wanted to automate the performance test setup we discussed in the last article. This includes a Logic App calling into a Cosmos DB stored procedure.

Logic App uses connectors when connecting to other services, either within or outside of Azure. Now how do we...
Invoking a Stored Procedure from a partitioned CosmosDB collection from Logic Apps
Solution · 19 Oct 2017
I struggled a little to make that work, so I thought I would share the learning in order to accelerate your future endeavour.

I was looking at a way to populate a CosmosDB quickly with random data.

Stored Procedures came to mind since they would skip client-server latency. We can call a stored procedure creating hundreds of documents with random...

Streaming

Synchronizing two streams with Kusto
Solution · 18 Mar 2020

We discussed Azure Data Explorer (ADX) and its query language Kusto in a past article.

In this article I want to talk about a typical problem in real time analytics: synchronizing two streams of data.

This happens all the time when sensor data are produced by different devices. Different devices may record measurements...
Event Hubs ingestion performance and throughput
Solution · 05 Jun 2018
Azure Event Hubs is a data streaming platform as a Service. It is an ingestion service.

We’ve looked at Event Hubs as the ingestion end of Azure Stream Analytics in two recent articles (here & here).

Here we look at client side performance of different...
Implementing & Automating Azure Stream Analytics Pipeline
Solution · 31 May 2018
In our last article, we set out to build a resilient architecture around streaming events.

In this article, we are going to build the solution. We are going to use an ARM template which automates deployment. We will also go through the configuration of different services.

The goal...
Taming the fire hose: Azure Stream Analytics
Solution · 22 May 2018
We came upon an interesting challenge recently.

Let’s have a stream of events pouring in. The source is unimportant. Hundreds, peeking at thousands, of events per second.

We need to update a table in Azure SQL with a summary of the events.

Let’s just assume the events are about...

Team

Corporate Cultures
Leadership · 09 Aug 2015
It is said that Netflix represents the new I.T. corporation well.

If you are interested in seeing what their corporate culture looks like, have a look a the slide deck they show to their job candidates.

It has all the flair of the typical Silicon...
Team IQ
Leadership · 15 Jul 2015

I’m still catching up on the articles I read months ago and wanted to share with you!

This one has been published in the New York Times and is about the intelligence of a team or the team IQ. It asks why some teams are...
50 Popular Business Books Summarized In One Sentence Each
News · 28 Jan 2015
Here is an excellent reading list of business books from Business Insider.

Each book is summarised in one snappy sentence that makes it for quick reference.

The list contains both books about how to succeed as a business or an individual in the business world.
Large Projects
Solution · 18 Nov 2014
There is something about large projects that you'll never find, hence never learn, in smaller projects. The complexity, both technical and in terms of people dynamics, creates an all new set of challenges.

I read the article I Survived an ERP Implementation – Top 10 Gems of Advice I Learned the Hard Way at the beginning of the...
Agile Trade-offs
Solution · 18 Jan 2013
It was very refreshing to read Paul Dolman-Darrall’s article about the trade-offs to adopt in an agile delivery model.

Agile is more than 10 years old but still have whimsical attributes of a brand new artefact. I believe this isn’t unique to agile but tend to be the case for any delivery methodology. It seems that however how...
Solution Skill Quadrants–Part II, Solution Architecture
Solution · 24 Nov 2011
In the last blog post I introduced my own patented Solution Skill model:

I won’t repeat the content of the last blog post, that’s what hyperlinks are for. The goal of this post is to address this simple question: as a Solution Architect, how can you help the situation?

…
Solution Skill Quadrants–Part I, the model
Solution · 22 Nov 2011
For quite a while I’ve been having a model of skill requirements in my head. That model guided me to analyse situations in different companies, mostly the staffing patterns for the maintenance of applications. I wanted to formalize it a little more and hence this post.

I call this model the Solution Skill Quadrants. Now you might ask what this...

Virtual Machines

Linux Custom Script - Docker Sandbox
Solution · 11 Apr 2018
I do a lot of proof of concepts (POCs) as part of my job.

I hate keeping demo environment around. They tend to become brittle, out-of-date and filled with the last stuff I did.

I prefer to start from a clean slate every single time when possible.

That means...
Finding a VM Image Reference, Publisher & SKU
Solution · 10 Jan 2018
Automation is great.

I love to script solutions in Azure.

But most of the time, that automation started with me fumbling and trialing different approaches in the portal. Once I got something around what I wanted I’ll automate.

I suspect you do the same thing.

This is why today we’re going...
Recurrent serverless batch job with Azure Batch
Solution · 20 Dec 2017
Most solutions have recurrent batch jobs, e.g. nightly / end-of-month batch jobs.

There are many services we can leverage in Azure to run those. In this article, we are going to explore a service that has “Batch” in its name: Azure Batch.

Azure Batch is typically positioned...
Disaster Recovery with VM Scale Sets & Geo-Replicated DBs
Solution · 09 Nov 2017
Last year we posted an article about different options available in Azure to implement a disaster recovery strategy.

We strongly suggest to review that article as it gives good insights about what a disaster recovery strategy is within an already resilient Cloud Environment but also clear out a...
Renaming Virtual Machine Disks
Solution · 17 Oct 2017
Let’s say we would like to rename disks on a Virtual Machine (VM). Here we mean renaming the Azure Resource Name of the managed disk. How would we go about that?

Why would we want to? Primarily to get our internal nomenclature right. A typical example is when we...
Azure Virtual Machines Anatomy
Solution · 16 Oct 2017
Virtual Machines can be pretty complex little beast. They can have multiple disks, multiple NICs in different subnets, can be exposed on the public internet either directly or through a load balancer, etc. .

In this article, we’ll look at the anatomy of a Virtual Machine (VM): what are...
Virtual Network Service Endpoint - Hello World
Solution · 04 Oct 2017
In our last post we discussed the new feature Virtual Network Service Endpoint.

In this post we’re going to show how to use that feature.

We’re going to use it on a storage account.

We won’t go through the micro steps of setting up each services but we’ll focus on the Service Endpoint configuration.

Resource Group

As usual for...
VNET Service Endpoints for Azure SQL & Storage
Solution · 02 Oct 2017
It’s finally here, it has arrived: Azure Virtual Network Service Endpoints.This was a long requested “Enterprise feature”.Let’s look at what this is and how to use it.Please note that at the time of this writing (end-of-September 2017) this feature is available only in a few region in Public Preview:<ul><li>Azure...
Moving from Standard to Premium disks and back
Solution · 27 Sep 2017
Azure Managed Disks (introduced in February 2017) simplified the way Virtual Machine disks are managed in Azure.

A little known advantage of that resource is that it exposes its storage type, i.e. Standard vs Premium, as a simple property that can easily be changed.

Why would we do that? Typically we’ll move from standard to premium storage to improve...
Sizing & Pricing Virtual Machines in Azure
Solution · 29 Mar 2017
I’m recurrently asked by customers similar questions around sizing & pricing of Virtual Machines (VMs), storage, etc. . So I thought I would do a reusable asset in the form of this article.

This is especially important if you are trying to size /...
Creating an image with 2 Managed Disks for VM Scale Set
Solution · 23 Feb 2017
UPDATE (23-06-2017): Fabio Hara, a colleague of mine from Brazil, has published the ARM template on his GitHub. This makes it much easier to try the content of this article. Thank you Fabio!

We talked about Managed Disks, now let’s use them.

Let’s create an image from an OS + Data disk & create a Scale Set with...

Taking a snapshot of a Managed Disk

Solution · 22 Feb 2017

We talked about Managed Disks, now let’s use them.

Let’s snapshot a Managed Disk and restore the snapshot on another VM.

Deploy ARM Template

{ "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "VM Admin User Name": { "defaultValue": "myadmin", "type": "string" }, "VM Admin Password": { "defaultValue": null, "type": "securestring" }, "VM Size": { "defaultValue": "Standard_DS4", "type": "string", "allowedValues":...

Migrating from unmanaged to managed disks
Solution · 21 Feb 2017
We talked about Managed Disks, now let’s use them.

Let’s migrate existing VMs from unmanaged to managed disks.

Install pre-requisites

Update your Microsoft Azure PowerShell to the latest bits in order to include Managed Disks functionalities

Deploy ARM Template
```
{ "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "VM Admin User Name": { "defaultValue": "myadmin", "type": "string" }, "VM Admin...
```
Azure Managed Disk–Overview
Solution · 20 Feb 2017

Microsoft released Azure Managed disk 2 weeks ago. Let’s look at it!

What did we have until then? The virtual hard disk (.vhd file) was stored as a page blob in an Azure Storage account.

That worked quite fine and Azure Disks are a little more than that. A...
Joining an ARM Linux VM to AAD Domain Services
Solution · 18 Jan 2017
Active Directory is one of the most popular domain controller / LDAP server around.

In Azure we have Azure Active Directory (AAD). Despite the name, AAD isn’t just a multi-tenant AD. It is built for the cloud.

Sometimes though, it is useful to have a traditional domain controller... in the cloud. Typically this is with legacy workloads built to work...
Troubleshooting NSGs using Diagnostic Logs
Solution · 09 Jan 2017
I’ve wrote about how to use Network Security Group (NSG) before.

Chances are, once you get a complicated enough set of rules in a NSG, you’ll find yourself with NSGs that do not do what you think they should do.

Troubleshooting NSGs isn’t trivial.

I’ll try to give some guidance here but...
Moving existing workloads to Azure
Solution · 12 Dec 2016

Applications born in the cloud can take full advantage of the cloud and the agility it brings.

But there are a lot of existing solutions out there that weren’t born in the cloud.

In this article I want to sketch a very high level approach on how...
Primer on Azure Monitor
Solution · 27 Nov 2016

Azure Monitor is the latest evolution of a set of technologies allowing Azure resources monitoring.

I’ve written about going the extra mile to be able to analyze logs in the past.

The thing is that once our stuff is in production with tons of users hitting...
Single VM SLA
Solution · 23 Nov 2016
By now you’ve probably heard the news: Azure became the first Public Cloud to offer SLA on single VM.

This was announced on Monday, November 21st.

In this article, I’ll quickly explore what that means.

Multi-VMs SLA

Before that announcement, in order to have SLA on connectivity...
Virtual Machine with 2 NICs
Solution · 23 Oct 2016
In Azure Resource Manager (ARM), Network Interface Cards (NICs) are a first class resource. You can defined them without a Virtual Machine.

UPDATE: As a reader kingly point out, NIC means Network Interface Controller, not Network Interface Card as I initially wrote. Don’t be fooled by the Azure logo ;)

Let’s...
Disaster Recovery with Azure Virtual Machines
Solution · 11 Jul 2016
UPDATE 09-11-2017: See Disaster Recovery with VM Scale Sets & Geo-Replicated DBs for an example of the different concepts introduced here.

UPDATE 31-05-2017: This article was written in 07-2016. Today Microsoft announced Azure Site Recovery (ASR) support for Azure-to-Azure (in public preview). Options presented in this article are still valid but the assumption that ASR can’t help no is longer...
Recreating VMs in Azure
Solution · 30 May 2016
In this article I’m going to explain how to destroy VMs, keep their disks on the backburner and re-create them later.

Why would you do that?

After all, you can shut down VMs and not be charged for it. You can later restart them and...

Web

Ingress rules in different Kubernetes namespaces
Solution · 11 Feb 2020

In this article I want to show how an ingress controller in Kubernetes can be used to route traffic to workloads deployed in multiple namespaces.

The online doc for AKS deploys everything in the same namespace. Hence this article is a thin extension to the online doc.

The basic trick is to deploy...
Azure App Service Authentication with Azure Front Door
Solution · 17 Jul 2019

Last time, we looked at Azure Front Door being used as a reverse proxy in front of Azure App Service.

We are going to increase the difficulty level today by looking at how to implement Azure AD authentication(also called easy auth) in Azure App Service behind Azure Front Door.

Although that...
Azure Front Door with App Service
Solution · 11 Jun 2019

Azure Front Door service was recently released.

Azure Front Door is an interesting service combining the capabilities of:
- Reverse Proxy (SSL Termination, URL based routing, URL rewrite & session affinity)
- Web Application Firewall (WAF)
- Accelerated Global routing
- Global Load Balancing between geo-distributed backend
- Some bits of Content Delivery Network (CDN, in the...
Azure Application Insights Role Name in web page
Solution · 06 Feb 2019

We discussed using Role Name to separate telemetries from different applications in a previous article.

We discuss the Nuget package I developed to set the role name for each telemetry in a backend application.

In this article I wanted to cover another element that receives little coverage: how to do that for...
Azure Application Insights Role Name Nuget Package
Solution · 23 Jan 2019

I’ve been using Azure Application Insights to monitor micro-services.

Since I have many micro-services, it isn’t practical to have an Application Insight resource per service. Given that those micro-services interact with each other and I want to monitor those interactions, it makes sense to get all telemetry in one place.

The issue with...
Understanding multiple Ingress in AKS
Solution · 28 Nov 2018

Last time, I covered the ins and outs of Ingress on AKS.

We looked at how to install nginx ingress controller and how the controller is deployed as a load balanced service. We did some URL based routing and domain name overloading.

This was in the spirit of clarifying the magic...
Understanding simple HTTP Ingress in AKS
Solution · 21 Nov 2018

We looked at Kubernetes Ingress conceptually. We looked at different use cases: URL based routing and multiple domains.

We also looked at how ingress was implemented from an AKS perspective, i.e. where traffic was routed in nodes.

In this article, I wanted to get hands on. I figured we could start...
Security with API: OAuth, token-based access vs key-based access
Solution · 11 Dec 2017
Let’s consider security with APIs, i.e how to securely identify the caller.

There are two authentication methods quite popular in the cloud to secure APIs:
1. Key-based access
2. OAuth, or token-based access in general
Let’s compare them.

Key-Based

By key-based we mean...
Azure Functions HTTP – Authorization Levels
Solution · 04 Dec 2017
In a past article, we looked at Serverless compute in Azure in general and Azure Functions specifically.

In this article we wanted to focus on Azure Function triggered by HTTP requests and the different options we have to authenticate:
- Anonymous
- Function
- Admin
- System
- User
Those are...
Using Application Gateway with VNET peering
Solution · 20 Nov 2017
As I write these lines (early November 2017), Application Gateway doesn’t integrate well across VNET peering.

More precisely, if we put the gateway in a VNET and have scale sets in another, the usual integration, where the scale set registers its VM automatically as the size of the set...
Azure Application Gateway Anatomy
Solution · 17 Jul 2017
Back in May, we talked about Azure Application Gateway.

In this article, we’re going to look at its anatomy, i.e. its internal component as exposed in the Azure Resource Manager (ARM) model.

A lot of Azure Resource has an internal structure. For instance, a Virtual Network has a collection of subnets.

Azure Application Gateway has a very rich internal...
URL Routing with Azure Application Gateway
Solution · 08 May 2017
Update (13-06-2017): The POC of this article is available on GitHub here.

I have a scenario perfect for a Layer-7 Load Balancer / Reverse Proxy:
- Multiple web server clusters to be routed under one URL hierarchy (one domain name)
- Redirect HTTP traffic to the same URL on HTTPS
- Have reverse proxy...
Web Socket Protocol
Solution · 21 Jan 2013
Web Socket is a new protocol, standardised in RFC 6455, attempting to bring the best features of HTTP & TCP together. More specifically, it aims at being Connected & full-duplex (as TCP), allowing server to call-back clients and universal (as HTTP).

This wasn’t done without pain. Web Socket has a non-trivial handshake process, done over HTTP, after which the...
Web Design: Make Complex thing Simple
Solution · 20 Nov 2012
I’m always amazed at you a simple user interface can simplify complex tasks.

Take Windows Explorer where you can drag & drop multiple files from one folder to another. The user sees which folders the files are going into, it takes a few seconds and boom! Doing that by command line would be much more abstract, left alone much more...
Internet Explorer 9 – Beta Update
News · 24 Nov 2010
An update to Internet Explorer 9 Beta is available from Microsoft as of yesterday (November 23rd 2010).

This is an update to the full browser as opposed to the developer preview build which isn’t the full Internet Explore, although the preview build does work side-by-side with any other version...
Microsoft’s jQuery contributions as official plug-ins
Solution · 04 Oct 2010
A quick note about Microsoft’s contributions to jQuery (namely XYZ): they are now official plug-ins!

We’ve read several months ago about the contributions of Microsoft to jQuery, that is, Globalization, Templates and Data Linking.

Those plug-ins bring a programming model relatively close to Microsoft WPF (or Silverlight) but quite in line with jQuery. It also shows the...

Tags

.NET

The Problem

API

Key-Based

Automation

The limits of ARM Templates

Big Data

Business

Containers

Online references