.NET
-
Beyond 2 concurrent connections in .NET
Solution ·I’m going to document this once and for all.
The Problem
![Ethernet-Cable-icon[1]](/assets/posts/2015/4/beyond-2-concurrent-connections-in-net/ethernet-cable-icon1.png)
You want to an endpoint multiple times in parallel. Or maybe you want to call multiple endpoints under the same domain name. For instance, you might want to drill an API with multiple requests because the API doesn’t...
-
Major upgrade to Azure DocumentDB LINQ provider
Solution ·![IC791289[1]](/assets/posts/2015/3/major-upgrade-to-azure-documentdb-linq-provider/ic7912891.png)
Early Septembre 2015, Microsoft has announced a major upgrade to the LINQ Provider of the .NET SDK of DocumentDb.I know it does appear a bit confusing since when DocumentDb was released (a year ago now), it was said that it supported SQL. Well, it supported some SQL.
Now...
-
DocumentDB Async Querying & Streaming
Solution ·UPDATE (31-08-2017): This article is superseded by the new article Cosmos DB Async Querying & Streaming.
Working with the .NET client SDK of Azure DocumentDB, I couldn’t find a way to query the store asynchronously.
***This post relates to the version 0.9.1-preview of Microsoft Azure DocumentDB Client Library. If you work with another major version, this might not...
-
Full Outer Join with LINQ to objects
Solution ·Quite a few times it happened to me to be looking for a way to perform a full outer join using LINQ to objects.
To give a general enough example of where it is useful, I would say ‘sync’. If you want to synchronize two collections (e.g. two collections of employees), then an outer join gives you a nice collection...
-
ePub Factory NuGet package
Solution ·I’ve been publishing this NuGet package.
Ok, so why do yet another ePub library on NuGet when there are already a few?
Well, there aren’t that many actually and none are Portable Class Library (PCL).
So I’ve built an ePub library portable to both Windows 8+ & .NET 4.5.1. Why not Windows Phone? My library is based...
-
The Missing Windows 8 Instructional Video
Gadget ·Scott Hanselman has produced an high quality video on YouTube.
The video is a nice and comprehensive introduction to Windows 8 for everyone (i.e. not geek only).
My experience with Windows 8 is that once you’ve figured out a few things (e.g. how to activate contextual Search), you can start appreciating the product. Before that, it just looks...
-
Entity Framework with Asynchronous behaviours
Solution ·They finally did it: the future release of Entity Framework (version 6) will sport asynchronous behaviour based on .NET 4.0 Task Parallel Library (TPL).
The API is pretty neat. First the SaveChanges gets an async brother SaveChangesAsync returning a Task. So we can now write things like:
await context.SavesChangesAsync();
The more complicated topic is the queries. ...
-
Windows Store App Lifecycle
Solution ·MSDN Magazine cool article about Windows Store App (aka Modern App, aka Metro Style app) lifecycle.
The author, Rachel Appel, raise the veil from how an app is managed by the OS in Windows 8.
Actually there are few states for an application to be in and few transitions but the devil is in the details!
<img title="How Windows...
-
Code Review in Visual Studio 2012
Solution ·The following article discusses the code review & annotation feature of Visual Studio 2012.
Finally integrated in Visual Studio & TFS, this feature is another step forward to integrate the entire Application Lifecycle Management (ALM) in one toolset.
-
Convertible Tablet: HP Envy x2
Gadget ·Finally some competition to Microsoft’s Surface hybrid!
[youtube http://www.youtube.com/watch?v=qHroxzEZfIY] <p>This goes in another direction (compared to Surface).</p> <p>This means the paradigm one-OS multiple hardware vendor is still good for consumers!</p>
-
Mocking Dependencies != IoC (or at least it doesn’t have to)
Solution ·Since I met the concept of Inversion of Control (IoC) I’ve been a big fan! What an elegant way to externalize dependencies and decouple a component from all the components it depends on. No longer is a component initializing a dependency initially coupling itself in a hard way.
A nice side-effect is that an IoC-compliant component is easier to...
-
Expression Trees Blog Series
Solution ·I’ve wrote a blog series around expression trees in .NET 4.0:
-
EF Power Tools CTP1 Released
Solution ·A little while ago, Microsoft released the first CTP of Entity Framework Power Tools.
Those power tools work on top of EF 4.1 and provide the following exciting features:
- Reverse Engineer a Code First Model from an existing database. Now this comes with the caveat that everything is configured through...
-
Expression Trees: Part 4 – Simple mapping inversion
Solution ·This is part of a series of blog post about expression trees in .NET 4.0. The past blog entries are:
In this article, I’ll cover simple mapping inversion using Expression Trees (ET). By simple I mean that we’ll map two objects properties / fields...
-
Expression Trees: Part 3 – Setting Properties
Solution ·This is part of a series of blog post about expression trees in .NET 4.0. The past blog entries are:
In this article, I’ll cover using Expression Trees (ET) for setting properties.
I’ll use the same object model I’ve used in the last article for examples:
public...
-
Expression Trees: Part 2 – Fetching Properties
Solution ·This is part of a series of blog post about expression trees in .NET 4.0. The past blog entries are:
In this article, I’ll cover using Expression Trees (ET) for fetching properties.
In this article, I’ll use the following classes:
public class Employee
{
public string FirstName { get; set;... -
Expression Trees: Part 1 – Basics
Solution ·I’m currently doing an SOA project and I have to put forward the solution architecture for many aspects of the solution. If you’ve done a bit of true SOA project (not to get into religious debate, let’s just say ‘where your architecture is oriented around services not just as a remote procedure call mechanism but where the services are first...
-
Entity Framework 4.1 – MSDN Article
Solution ·There is a comprehensive article about Entity Framework 4.1 on MSDN Magazine of May 2011:
http://msdn.microsoft.com/en-ca/magazine/hh126815.aspx
You can also read my blog series on the topic.
-
Entity Framework 4.1 Series
Solution ·I did a few Proof of Concepts (POC) around the new Entity Framework 4.1 as part of an SOA project I’m currently on and I decided to share the acquired knowledge. The following series of blog posts are the result:
-
Entity Framework 4.1: Bypassing EF query mapping (8)
Solution ·This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
- Basics (1)
- Override conventions (2)
- Deep Fetch vs Lazy Load (3)
- Complex Types (4)
- Many to Many Relationships (5)
- Optimistic Concurrency (6)
- Inheritance (7)
In this...
-
Entity Framework 4.1: Inheritance (7)
Solution ·This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
- Basics (1)
- Override conventions (2)
- Deep Fetch vs Lazy Load (3)
- Complex Types (4)
- Many to Many Relationships (5)
- Optimistic Concurrency (6)
In this article, I’ll cover inheritance.
…
-
Entity Framework 4.1: Optimistic Concurrency (6)
Solution ·This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
- Basics (1)
- Override conventions (2)
- Deep Fetch vs Lazy Load (3)
- Complex Types (4)
- Many to Many Relationships (5)
In this article, I’ll cover optimistic concurrency.
Very often we need to manage many-users concurrency. This...
-
Entity Framework 4.1: Many to Many Relationships (5)
Solution ·This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
In this article, I’ll cover the many to many relationships.
Let’s start with the easiest case, we’ll let EF...
-
Entity Framework 4.1: Complex Types (4)
Solution ·This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
In this article, I’ll cover the complex types.
By default, EF 4.1 is mapping classes to table. That is convenient but sometimes, we...
-
Entity Framework 4.1 Release to Web (RTW)
Solution ·Two weeks ago, Microsoft published a Release Candidate (RC) of Entity Framework 4.1. They promised to release the product within 4 weeks.
Well, they released it today!
So you can now use Entity Framework 4.1 which is a fully supported Microsoft Product.
The only change done from the RC is the default length of nvarchar from 128...
-
Entity Framework 4.1: Deep Fetch vs Lazy Load (3)
Solution ·This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
In this article, I’ll cover the control of what is getting loaded in queries.
EF 4.1 is able to manage relations. Now which relations get loaded when you do a query? ...
-
Entity Framework 4.1: Override conventions (2)
Solution ·This is part of a series of blog post about Entity Framework 4.1. The past blog entries are:
In this article, I’ll cover how to over conventions.
We’ve seen that EF 4.1 Code-First infer the mapping between the model (classes) and the tables according to conventions. When those conventions aren’t to our liking, we...
-
Entity Framework 4.1: Basics (1)
Solution ·As I wrote a few days ago, I’ve started to toy around with Entity Framework 4.1, more specifically around the features of Code First.
I’m currently consulting for an SOA project and I have to put forward the solution architecture for many aspect of the solution, including the data access layer. So I did quite a few POCs in...
-
Entity Framework 4.1 RC: Including Code-First
Solution ·Microsoft has published the first release candidate of Entity Framework 4.1. This new release includes the very popular Code First feature, allowing to model your data in the code, without any designer tools.
With this Release Candidate comes a promise: the final release will come within the next 4 weeks and will be %100 compatible with the current RC.
…
-
Asynchrony in .NET
Solution ·Microsoft recently released a whitepaper on the future of Asynchrony in .NET along with a CTP SDK.
This CTP attempts to address one concern: how to make asynchrony easy in .NET by removing all the friction in code.
In .NET, Asynchrony, so far, as been treated with the pattern BeginXYZ / EndXYZ dynamic duo. For instance, in the...
-
Entity Framework Code First CTP 5
Solution ·I wrote a blog post a while ago about Entity Framework’s POCO capability. Basically, the current state is that you don’t have POCO. You have a T4 template to generate classes without EF attributes or base classes, but it stops short of being a real POCO implementation once you start looking at the collections.
Microsoft is working on a...
-
Introduction to TPL Dataflow
Solution ·In Octobre 2010, Microsoft released a white paper on an oncoming library in the .NET Framework, the Task Parallel Library DataFlow.
The dataflow library is built on top of the Task Parallel Library (TPL) included in .NET 4.0. Basically, the TPL provides the building block for task-oriented Frameworks. The dataflow...
-
Entity Framework Patterns: Select Multiple Entities
Solution ·I just published another new article on code project:
http://www.codeproject.com/KB/linq/EFPatternsSelectMultiples.aspx
It’s still pending for reviews.
I thought I could start teasing out some Entity Framework patterns for non-trivial scenarios.
In this case, I’m trying to address the scenario where you want to select a given list of entities (you’re given the list of IDs of those entities). I...
-
Entity Framework 4.0: POCO or POCO?
Solution ·The Entity Framework shipping with .NET Framework 4.0 is a major improvement over .NET Framework 3.5 SP1 Entity Framework, which was the first version. The first version was a nice curiosity but had many shortfall. For instance, it didn’t support stored procedures, the designer wasn’t flexible, the produced ad hoc queries were quite hard to read, etc. .
EF...
-
Scott Gu: Silverlight is strategic for Microsoft
Solution ·
The Gu has spoken: Silverlight is important and strategic for Microsoft.A bit like I mentioned earlier this week, the main points of Microsoft strategy are that Silverlight is for rich client applications while HTML 5 is for broader reach.
An interesting point of Scott Gu is that many of...
-
Microsoft Silverlight Strategy
Solution ·
Apparently I wasn’t the only one to notice the lack of vigour behind the Silverlight marketing machine at the PDC last week. Indeed, that and Bob Moglia’s declarations triggered one of those cyber-tsunami internet is famous for.
Moglia, Microsoft President in charge of server and tools business, made an <a...
-
HTML 5 or Silverlight?
Solution ·Ah, this is a tough question!
As I mentioned yesterday, I was in Microsoft Office in Montreal for the PDC keynote diffusion after which there was a discussion about the keynote.
The room was filled mainly with freelancers and small company consultants (that is quite typical for a Montreal crowd actually). More than half of those individuals are Microsoft...
API
-
Accessing Azure Key Vault from within Azure API Management
Solution ·
Today we look at a common although slightly advanced scenario with API Management: accessing Azure Key Vault from Azure API Management.
In an Enterprise, API Management service are often shared between teams. This means a lot of people might open it in the Portal and look at it. It also means that putting...
-
Tracing request in Azure API Management
Solution ·
Azure API Management (API-M) is an API Gateway solution.
It is quite easy to create an API in API-M, connect it to a back-end API and test it in the portal.
What happen when you test it from another client platform (e.g. Postman) and it fails? You have no visibility into what...
-
API Management - OAuth and private back-ends
Solution ·
Azure API Management is a fully managed API Gateway service.
In my last article we looked at the anatomy of the service. Today I wanted to demonstrate how to use OAuth with JWT token to protect an API Front End. I also wanted to show how we can access backend APIs...
-
Anatomy of API Management
Solution ·
When I want to wrap my head around a non-trivial Azure service with a few moving parts, I like to draw a diagram.
That might be from my UML days or it might just be that I reason better with boxes that concepts spread in documentation and APIs. In general, it helps me...
-
How to use Azure Data Lake Storage REST API
Solution ·
Azure Data Lake Storage (ADLS) Generation 2 has been around for a few months now.
That new generation of Azure Data Lake Storage integrates with Azure Storage. This makes it a service available in every Azure region. It also makes it easier to access as it is built on foundation well known...
-
API Management exclusive access to Azure Function
Solution ·
[Update 05-04-2019: Erratum on the original article. Logic Apps is actually able to perform public IP filering.]
Azure API Management acts as a front door to your APIs.
Typically, we do not want users / apps to be able to access the underlying APIs directly since that would bypass the API Management policies,...
-
Accessing Azure Key Vault using Managed Service Identity Logic Apps
Solution ·
Azure Key Vault is a great service to manage secrets, keys & certificates.
It uses RBAC to control access. Like all access control system, there is a chain of access. For instance, my user account has access to the vault: this means if my account’s credentials get leaked, the access to the...
-
Using Azure DevOps REST API to start multiple releases with Azure Logic Apps
Solution ·
In our last article, we looked at how we could leverage the Azure DevOps REST API to trigger multiple releases.
This is useful when rebuilding an environment: first we release the shared infrastructure then each service on top of it.
In this article, we’ll see an implementation example using Azure Logic...
-
Using Azure DevOps REST API to start multiple releases
Solution ·
Azure DevOps CI / CD (i.e. Build / Release) mechanism is very valuable to me. It makes things reproduceable & robust. It allows me to change whatever I want since I have this constant safety net underneath.
For simple projects, having a build and release is fine. With micro-services I adopt the pattern of...
-
How to get rid of /api in Azure Function’s route?
Solution ·
We looked at Azure Functions.We also looked at security around Azure Function used to implement APIs.
Something people will quickly notice when implementing an Webhook / API function is that its URL or route is always prepended by /api.
For instance, if we create a webhook...
-
Security with API: OAuth, token-based access vs key-based access
Solution ·![pexels-photo-326569[1]](/assets/posts/2017/4/security-with-api-oauth-token-based-access-vs-key-based-access/pexels-photo-3265691_thumb.jpg "pexels-photo-326569[1]")
Let’s consider security with APIs, i.e how to securely identify the caller.There are two authentication methods quite popular in the cloud to secure APIs:
- Key-based access
- OAuth, or token-based access in general
Let’s compare them.
Key-Based
By key-based we mean...
-
Azure Functions HTTP – Authorization Levels
Solution ·![pexels-photo-277593[1]](/assets/posts/2017/4/azure-functions-http-authorization-levels/pexels-photo-2775931_thumb.jpg "pexels-photo-277593[1]")
In a past article, we looked at Serverless compute in Azure in general and Azure Functions specifically.In this article we wanted to focus on Azure Function triggered by HTTP requests and the different options we have to authenticate:
- Anonymous
- Function
- Admin
- System
- User
Those are...
-
Nuget WordPress REST API – Demo App
Solution ·I’ve had a few requests to explain how to use the Nuget WordPress REST API beyond authentication.
In order to do this, I added a Demo App under source control.
The Nuget package source code is available at http://wordpressrestapi.codeplex.com/ and if you download the code, you’ll see there are 3 projects:
- WordpressRestApi: essentially the Nuget package ...
-
Refactoring Tags in WordPress Blog
Solution ·
I did refactor the tags of my blog this week end!I display the tags (as a word cloud) on the right-hand side of my pages. The tags grew organically since I started blogging in 2010.
As with many things that grow organically, it got out of hand with time. ...
-
Securing REST API using Azure Active Directory
Solution ·Scenario: you have a web & mobile front-end, both using a REST API as a back-end. You want to secure that back-end with authentication / authorization. How do you do that in Azure?
There are obviously a bunch of ways to do that. In this post, I’ll discuss the recommended...
-
Docker Containers on Windows Server
Solution ·If you had any doubts about the increased pace in IT innovation, look at Docker Containers. The project was open sources in March 2013 as a container technology for Linux and 1.5 years later, in Octobre 2014, Microsoft announced they were integrating that technology on Windows Server 2016!
That’s 1.5 years from toe in the water to major...
-
Nuget WordPress REST API - On Codeplex!
Solution ·I’ve introduced WordPress REST API Nuget package in my last blog post.
It’s an SDK to access Wordpress REST API.
![nugetlogo[1]](/assets/posts/2015/3/nuget-wordpress-rest-api-on-codeplex/nugetlogo1.png)
Today, I’ve open sourced it on Codeplex.You can visit it on https://wordpressrestapi.codeplex.com/.
If you want to collaborate to increase the API...
-
Nuget Wordpress REST API – Authentication
Solution ·![wordpress_logo[1]](/assets/posts/2015/3/nuget-wordpress-rest-api-authentication/wordpress_logo1_thumb.jpg "wordpress_logo[1]")
I use Wordpress.com as my blog platform. It hosts the Wordpress CMS software and adds a few goodies.I was curious about their API after noticing that my Blog App (Windows Live Writer) tended to create duplicate of pictures, leaving lots of unused assets...
-
SOA vs Mobile APIs
Solution ·I recently read an article from Bill Appleton of Dream Factory with the provocative title SOA is not a Mobile Backend.
It raised quite a few good points that were in the back of my mind for more than a year.
Basically, what is the difference between SOA and API?
-
Azure Key Vault
Solution ·Has somebody been peeking on my X-mas list?
Indeed, one of the weakness of the current Azure Paas solution I pointed out last year was that on non-trivial solutions you end up with plenty of secrets (e.g. user-name / password, SAS, account keys, etc.) stored insecurely in your web.config (or similar store).
I was suggesting, as a solution, to...
-
Surveying the Web API Security Landscape
Solution ·Ronnie Mitra published a nice review of the current security protocols & standards available to you to protect your API.
Ronnie gives a good overview of what used to be available, in SOAP WS-*, what is currently available (HTTPS, OAuth and the like) and what is brewing for the future (Oz & Macaroons).
-
Description of your REST API: Swagger & RAML
Solution ·About a year ago I did a post about WADL. I hope you didn’t build a billion dollars business model around that initiative because it hasn’t left the ground yet :(
WADL, or Web Application Description Language for the initiated, was aimed to be what WSDL is for SOAP Web Services but for REST.
You see, the thing with...
-
Large Projects
Solution ·There is something about large projects that you'll never find, hence never learn, in smaller projects. The complexity, both technical and in terms of people dynamics, creates an all new set of challenges.
I read the article I Survived an ERP Implementation – Top 10 Gems of Advice I Learned the Hard Way at the beginning of the...
-
WADL in a bottle eating noodles
Solution ·In my last entry about REST web services I talked about its biggest weakness for me: the lack of description model of REST services.
The idea of hitting an HTTP endpoint as a shot in the dark is for me quite a leap of faith, and very likely an invitation to spend hours troubleshooting.
But despair no more, enters...
-
REST style with Hypermedia APIs
Solution ·Once upon a time there was SOAP. SOAP really was a multi-vendor response to CORBA. It even share the same type of acronym, derived from object. Objects are so 90’s dude… The S in SOAP stands for Simple by the way. Have a go at a bare WSDL and try to repeat in your head that it is...
-
Applied SOA Series
Solution ·I’ve written (a year and a half ago actually) a blog series around all the concerns and issues around implementing SOA in an Enterprise Solution:
-
Applied SOA: Part 9–Service Versioning
Solution ·This is part of a series of blog post about Applied SOA. The past blog entries are:
- Introduction
- SOA Basics
- Service Discovery Process
- Service Taxonomy
- Interoperability
- Service Composition
- System Consistency
- Security
In this article, I’ll cover Service Versioning.
In SOA a Service...
-
Applied SOA: Part 8–Security
Solution ·This is part of a series of blog post about Applied SOA. The past blog entries are:
- Introduction
- SOA Basics
- Service Discovery Process
- Service Taxonomy
- Interoperability
- Service Composition
- System Consistency
In this article, I’ll cover security. Security is a very broad topic and by...
-
Applied SOA: Part 7–System Consistency
Solution ·This is part of a series of blog post about Applied SOA. The past blog entries are:
- Introduction
- SOA Basics
- Service Discovery Process
- Service Taxonomy
- Interoperability
- Service Composition
In this article, I’ll cover system consistency. In every distributed solution, a major...
-
Applied SOA: Part 6–Service Composition
Solution ·This is part of a series of blog post about Applied SOA. The past blog entries are:
In this article, I’ll quickly cover service composition.
Service Composition is often stated as a goal of SOA. I...
-
Integration in the Cloud
Solution ·I ran across a series of good blog posts I wanted to share with you, dear readers.
From Richard Seroter, the series is about integration patterns in the cloud, looking at cloud-to-cloud & cloud-to-on-premise scenarios.
Richard looks at three main patterns:
For each...
-
Applied SOA: Part 5 – Interoperability
Solution ·This is part of a series of blog post about Applied SOA. The past blog entries are:
In this article, I’ll cover the Interoperability. SOA aims is to expose capabilities through services in order...
-
Applied SOA: Part 4 – Service Taxonomy
Solution ·This is part of a series of blog post about Applied SOA. The past blog entries are:
In this article, I’ll cover the Service Taxonomy. On an SOA project, we’re going to have services. How should we classify...
-
Applied SOA: Part 3 – Service Discovery Process
Solution ·This is part of a series of blog post about Applied SOA. The past blog entries are:
In this article, I’ll cover the Service Discovery Process. You’re working on an SOA project. There’s going to be services. What are those services?
That sounds...
-
Applied SOA: Part 2 – SOA Basics
Solution ·SoThis is part of a series of blog post about Applied SOA. The past blog entries are:
In this article, I’ll cover the basics of SOA.
The hardest question for an Architect to answer briefly is ‘what is architecture’. Probably the second hardest question is ‘what is SOA’? In both cases, if you receive a...
-
WCF Express Interop Bindings
Solution ·Something that might come as a surprise for somebody who worked with WCF (even for years) but never had to interoperate with another platform than .NET is how much WCF isn’t interoperable out-of-the-box.
It isn’t WCF fault really. It’s SOAP’s fault and its lousy specs. Well… not so much lousy as big, complicated and extremely flexible. The result is that...
-
Applied SOA: Part 1 – Introduction
Solution ·I’ve been involved in multiple projects sporting some form of Service Oriented Architecture using Microsoft .NET technologies. Some were SOA in name only (Remote Procedure Calls were involved) while others were closer in spirit to the tenets of SOA.
The term SOA tends to be abused in the community. When I do interviews with candidates, I often see SOA on...
-
Globalization and web services
Solution ·I always found that globalization is the poor child of computer literature. In any .NET book, you won’t hear specifically about globalization before chapter 23. You can even tell it’s not shooting high on the radar when you look at the ways globalization was implemented in the three main front-end Frameworks of .NET: WinForm (great!), ASP.NET (retrofitted in sub-optimal way...
-
Globalization Patterns in WCF (WS-I18N implementation)
Solution ·I recently came across a good Code Project article:
Globalization Patterns in WCF (WS-I18N implementation)
written by Pablo M. Cibraro.
Basically, there exists a standard specs (WS-I18N) describing how to pass international information (e.g. locale & time zone) to a SOAP endpoint in order for that web service to return you localized data.
An interesting aspect...
-
Using WCF for exposing and consuming OData
Solution ·I just published a new article on CodeProject:
http://www.codeproject.com/KB/webservices/WCFDataServices.aspx
It hasn’t been reviewed yet.
In that article I do showcase different ways to use OData to expose your data using WCF Data Services. The samples grow in complexity, going from the standard hello world of WCF Data Services (ie exposing your DB on a web endpoint), to more...
-
Sharing Data Contracts between clients & servers with WCF Data Services
Solution ·I’ve been blogging a bit about the OData protocol put forward by Microsoft and even wrote an article about it on Code Project. That article is supposed to be followed by others about WCF Data Services, the .NET implementation of OData, well… stay tuned!
I love...
-
Consuming FaultContract with Restful WCF services
Solution ·I’ve learned something new this morning while doing a proof-of-concept around WCF REST services.
I did a little WPF demo app both hosting and consuming web services. I was trying to handle faults properly, using fault contracts. I had experience with SOAP-based WCF web services, so I went the motions, starting by creating a fault contract:
[DataContract<a...
-
Overview of OData on Code Project
Solution ·I’ve contributed an entry-level article on code project about the Open Data Protocal (OData).
I go through the basics of the protocol, how it is structured, how to consume it and where I see usage.
I did this in order to follow suit with an article on ADO.NET around OData and I didn’t have anything to point at for foundation!
…
![pexels-photo-326569[1]](/assets/posts/2017/4/security-with-api-oauth-token-based-access-vs-key-based-access/pexels-photo-3265691.jpg "pexels-photo-326569[1]")
![pexels-photo-277593[1]](/assets/posts/2017/4/azure-functions-http-authorization-levels/pexels-photo-2775931.jpg "pexels-photo-277593[1]")
![wordpress_logo[1]](/assets/posts/2015/3/nuget-wordpress-rest-api-authentication/wordpress_logo1.jpg "wordpress_logo[1]"){kind=logo}
Automation
-
API Management - OAuth and private back-ends
Solution ·Azure API Management is a fully managed API Gateway service.
In my last article we looked at the anatomy of the service. Today I wanted to demonstrate how to use OAuth with JWT token to protect an API Front End. I also wanted to show how we can access backend APIs...
-
Deploying an ARM template hierarchy
Solution ·
ARM templates are a powerful way to deploy Azure resources.
It’s a declarative language which express the target state. It can therefore be used to create or update resources.
An ARM template is a JSON file. Once a target is big enough it becomes convenient to split the ARM template into multiple files....
-
Using Azure DevOps REST API to start multiple releases with Azure Logic Apps
Solution ·In our last article, we looked at how we could leverage the Azure DevOps REST API to trigger multiple releases.
This is useful when rebuilding an environment: first we release the shared infrastructure then each service on top of it.
In this article, we’ll see an implementation example using Azure Logic...
-
Using Azure DevOps REST API to start multiple releases
Solution ·Azure DevOps CI / CD (i.e. Build / Release) mechanism is very valuable to me. It makes things reproduceable & robust. It allows me to change whatever I want since I have this constant safety net underneath.
For simple projects, having a build and release is fine. With micro-services I adopt the pattern of...
-
Azure Dev Ops CI / CD pipelines with AKS - Lessons learned
Solution ·
I have this personal project where I have a swag of micro services collaborating to serve different web applications.
I started to work on the Continuous Integration / Continuous Deployment (CI / CD) using Azure DevOps.
I did toy around those areas before. But this was really a rubber hits the road experience...
-
Service Principal for Logic App Connector
Solution ·
Azure Logic Apps is a powerful integration platform.
It integrates with different services (inside and outside Azure) using connectors.
Connectors are responsible to authenticate to the service they represent.
Some connectors will hold the credentials. This is the case, for instance, of the SQL connector.
Other connectors will by default take the...
-
Deploying AKS with ARM Template – Network integration
Solution ·
In a past article, we looked at how Azure Kubernetes Services (AKS) integrated with Azure Networking.
AKS is a managed Kubernetes service in Azure.
In this article, we are going to do two things:
- Deploy an AKS cluster with Advanced Networking using an Azure ARM Template.
- Deploy a service on the...
-
Creating a Service Principal with Azure CLI
Solution ·
Service Principals are a bit of a weird beast. They are Azure Active Directory applicationswith kind of an extra bit. That bit says they can actually login by themselves. Hence the name principal. But being an application is kind of weird.
We covered Service Principals in the past. We covered how to...
-
RBAC and role assignment using ARM Templates
Solution ·
Azure supports Role Based Access Control (RBAC) as an access control paradigm.It allows to map a user (or a group of users) to a role within a given scope (resource, resource group, subscription or management group).
For instance, we could map my user identity to a Virtual...
-
Cosmos DB Configuration Management
Solution ·
I often found the need to automate Cosmos DB deployments.I like to have users run an ARM template and be ready to go. Automation is key component to repeatability. It enables automated testing and increase agility in many other ways.
ARM Templates don’t go beyond a database account.
…
-
Using Azure Container Instances to complete automation
Solution ·
We looked at Azure Container Instances.It allows us to launch a container without a cluster and without a VM.
It is ideal for bursting scenarios.
In this article we’ll look at another scenario for container instances: automation.
The limits of ARM Templates
A great tool for automation...
-
Implementing & Automating Azure Stream Analytics Pipeline
Solution ·
In our last article, we set out to build a resilient architecture around streaming events.In this article, we are going to build the solution. We are going to use an ARM template which automates deployment. We will also go through the configuration of different services.
The goal...
-
Linux Custom Script - Docker Sandbox
Solution ·
I do a lot of proof of concepts (POCs) as part of my job.I hate keeping demo environment around. They tend to become brittle, out-of-date and filled with the last stuff I did.
I prefer to start from a clean slate every single time when possible.
That means...
-
Finding a VM Image Reference, Publisher & SKU
Solution ·
Automation is great.I love to script solutions in Azure.
But most of the time, that automation started with me fumbling and trialing different approaches in the portal. Once I got something around what I wanted I’ll automate.
I suspect you do the same thing.
This is why today we’re going...
-
Setup for populating Cosmos DB with random data using Logic Apps
Solution ·![pexels-photo-267968[1]](/assets/posts/2017/4/setup-for-populating-cosmos-db-with-random-data-using-logic-apps/pexels-photo-2679681_thumb.jpg "pexels-photo-267968[1]")
We recently published an article about Cosmos DB Performance with Geospatial Data.In this article, we’re going to explain how to setup the environment in order to run those performance test.
More importantly, we believe this article is interesting on its own as it shows how to use...
-
How to Create a Logic App Connector in an ARM Template
Solution ·![pexels-photo-415043[1]](/assets/posts/2017/4/how-to-create-a-logic-app-connector-in-an-arm-template/pexels-photo-4150431_thumb.jpg "pexels-photo-415043[1]")
I wanted to automate the performance test setup we discussed in the last article. This includes a Logic App calling into a Cosmos DB stored procedure.Logic App uses connectors when connecting to other services, either within or outside of Azure. Now how do we...
-
How to know where a Service is Available in Azure
Solution ·![pexels-photo-269633[1]](/assets/posts/2017/3/how-to-know-where-a-service-is-available-in-azure/pexels-photo-2696331_thumb.jpg "pexels-photo-269633[1]")
Azure has a Global Footprint of 40 regions at the time of this writing (mid-September 2017).Not all services are available in every regions. Most aren’t in fact. Only foundational services (e.g. storage) are available everywhere.
In order to know where a service is available, we...
-
Creating an image with 2 Managed Disks for VM Scale Set
Solution ·UPDATE (23-06-2017): Fabio Hara, a colleague of mine from Brazil, has published the ARM template on his GitHub. This makes it much easier to try the content of this article. Thank you Fabio!
We talked about Managed Disks, now let’s use them.
Let’s create an image from an OS + Data disk & create a Scale Set with...
-
Automating Azure AD
Solution ·
In the previous article, we explored how to interact (read / write) to an Azure AD tenant using Microsoft Graph API.
In the article before that, we looked at how to authenticate a user without using Azure AD web flow.
Those were motivated by a...
-
Azure SQL Elastic Pool – Moving databases across pools using PowerShell
Solution ·![hand-truck-564242_640[1]](/assets/posts/2017/1/azure-sql-elastic-pool-moving-databases-across-pools-using-powershell/hand-truck-564242_6401_thumb.jpg "hand-truck-564242_640[1]")
I’ve written a bit about Azure SQL Elastic Pool lately: an overview, about ARM template and about database size.
One of the many great features of Azure SQL Elastic Pool is that like Azure SQL Database (standalone), we can change the eDTU capacity of the pool...
-
Azure SQL Elastic Pool – Database Size
Solution ·
I mentioned in a past article, regarding database sizes within an elastic pool:“No policies limit an individual database to take more storage although a database maximum size can be set on a per-database basis.”
I’m going to focus on that in this article.
An Azure SQL Database resource has...
-
Azure SQL Elastic Pool - ARM Templates
Solution ·![coil-632650_640[1]](/assets/posts/2016/4/azure-sql-elastic-pool-arm-templates/coil-632650_6401_thumb.jpg "coil-632650_640[1]")
In my last article, I covered Azure SQL Elastic Pool. In this one I cover how to provision it using ARM templates.As of today (December 2016), the documentation about Azure SQL Elastic Pool provisioning via ARM templates is… not existing.
Searching for it I was able...
-
Primer on Azure Monitor
Solution ·![pexels-photo-42384[1]](/assets/posts/2016/4/primer-on-azure-monitor/pexels-photo-423841_thumb.jpg "pexels-photo-42384[1]")
Azure Monitor is the latest evolution of a set of technologies allowing Azure resources monitoring.
I’ve written about going the extra mile to be able to analyze logs in the past.
The thing is that once our stuff is in production with tons of users hitting...
-
I tortured Azure in the Week-End
Solution ·In my last blog post, I showed how to author an Azure Runbook to turn VMs up and down.
In order to test it I did put a schedule where at every hour (e.g. 1:00, 2:00, 3:00, etc.) it would start the VMs and at every pass half hour (e.g. 1:30, 2:30, 3:30, etc.).
The thing is that I...
-
Shutting down VMs on schedule in Azure
Solution ·I thought it was time for a post on the quintessential automation task: shutting down VMs & starting them on a schedule.
UPDATE (Nov-27th 2016): this particular task is now available directly in the VM portal (at least the shut down part of it). This article remains interesting to look at a complete example of Azure Automation.
This is a...
-
Azure Runbook - A complete (simple) example
Solution ·I meant to write about Azure Runbooks (also known as Azure Automation) for quite a while.
I had the chance to be involved in the operations of a solution I helped architect. When you get beyond trivial Azure Solution, like on premise, you’ll want to have some automations. For instance, you’ll want to:
- Clean up data at the...
![pexels-photo-267968[1]](/assets/posts/2017/4/setup-for-populating-cosmos-db-with-random-data-using-logic-apps/pexels-photo-2679681.jpg "pexels-photo-267968[1]")
![pexels-photo-415043[1]](/assets/posts/2017/4/how-to-create-a-logic-app-connector-in-an-arm-template/pexels-photo-4150431.jpg "pexels-photo-415043[1]")
![pexels-photo-269633[1]](/assets/posts/2017/3/how-to-know-where-a-service-is-available-in-azure/pexels-photo-2696331.jpg "pexels-photo-269633[1]")
![hand-truck-564242_640[1]](/assets/posts/2017/1/azure-sql-elastic-pool-moving-databases-across-pools-using-powershell/hand-truck-564242_6401.jpg "hand-truck-564242_640[1]")
![coil-632650_640[1]](/assets/posts/2016/4/azure-sql-elastic-pool-arm-templates/coil-632650_6401.jpg "coil-632650_640[1]")
![pexels-photo-42384[1]](/assets/posts/2016/4/primer-on-azure-monitor/pexels-photo-423841.jpg "pexels-photo-42384[1]")
Big Data
-
Python Version in Databricks
Solution ·In the last few months, we’ve looked at Azure Databricks:
- Getting Started
- Resilient Distributed Dataset
- Spark SQL – Data Frames
- Transforming Data Frames in Spark
- Parsing escaping CSV files in Spark
- Import Notebooks in Databricks
In those articles, we used the Python SDK (also... -
Import Notebooks in Databricks
Solution ·
We’ve recently looked Azure Databricks:
- Getting Started
- Resilient Distributed Dataset
- Spark SQL – Data Frames
- Transforming Data Frames in Spark
- Parsing escaping CSV files in Spark
In most cases we did share notebooks on GitHub.
Here we wanted to show...
-
Azure Databricks - Parsing escaping CSV files in Spark
Solution ·
In previous weeks, we’ve looked at Azure Databricks, Azure’s managed Spark cluster service.We then looked at Resilient Distributed Datasets (RDDs) & Spark SQL / Data Frames. We also looked at an example of more tedious transformation prior to querying using the H-1B Visa...
-
Azure Databricks - Transforming Data Frames in Spark
Solution ·
In previous weeks, we’ve looked at Azure Databricks, Azure’s managed Spark cluster service.We then looked at Resilient Distributed Datasets (RDDs) & Spark SQL / Data Frames.
We wanted to look at some more Data Frames, with a bigger data set, more precisely some...
-
Azure Databricks - Spark SQL - Data Frames
Solution ·
We looked at Azure Databricks a few weeks ago.Azure Databricks is a managed Apache Spark Cluster service.
More recently we looked at how to analyze a data set using Resilient Distributed Dataset (RDD). We used the Social characteristics of the Marvel Universe public dataset,...
-
Azure Databricks - RDD - Resilient Distributed Dataset
Solution ·![pexels-photo-756901[1]](/assets/posts/2018/1/azure-databricks-rdd-resilient-distributed-dataset/pexels-photo-7569011_thumb.jpg "pexels-photo-756901[1]")
We looked at Azure Databricks a few weeks ago. Azure Databricks is a managed Apache Spark Cluster service.In this article, we are going to look at & use a fundamental building block of Apache Spark: Resilient Distributed Dataset or RDD. We are going to use the...
-
Azure Databricks - Getting Started
Solution ·![Databricks_logo[1]](/assets/posts/2017/4/azure-databricks-getting-started/databricks_logo1_thumb.png "Databricks_logo[1]")
Apache Spark is rising in popularity as a Big Data platform. It exists on this accelerated timeline for such an impactful technology.Think about it:
- 2009, started as a Berkeley’s University project.
- 2010, open sourced
- 2013, donated to Apache Foundation
- 2014, becomes...
-
Querying tables in Azure Data Lake Analytics
Solution ·Azure Data Lake (both Storage & Analytics) has been in public preview for a month or two.
You can get started by reading this or have a look at Azure Data Lake series for more posts on Azure Data Lake.
I thought I would kick some posts about more complex scenarios to display...
-
Network Access Control on an HDInsight Cluster
Solution ·In this post, I’m gona try to show how you can use an Azure Virtual Network with a Network Security Group to control the access (at the network level) to an HDInsight cluster.
For a primer on both those technologies, please refer to my Using Network Security Groups (NSG) to secure network access to an environment post.
The...
-
Analyzing Web Logs with Azure Data Lake Analytics (ADLA)
Solution ·Azure Data Lake (both Storage & Analytics) has been in public preview for a month or two.
You can get started by reading this or have a look at Azure Data Lake series for more posts on Azure Data Lake.
I thought I would kick some posts about more complex scenarios to display...
-
Stategic Data, Infonomics & Big Data
Solution, Strategy ·UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.
I wanted to talk a bit about your Strategic Data & the concept of Data Lake (regardless of its implementation).Nowaday, data is seen less and less as...
-
Azure Data Lake Analytics - Loading files with custom C# code
Solution ·UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.
Azure Data Lake (both Storage & Analytics) has been in public preview for a month or two.
You can get started by reading this.
I thought I would kick some posts about more complex scenarios to...
-
U-SQL Cross Apply
Solution ·UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.
Azure Data Lake (both Storage & Analytics) has been in public preview for a month or two.
You can get started by reading this.
I thought I would kick some posts about more complex scenarios...
-
Registering assemblies in Azure Data Lake Analytics
Solution ·UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.
Azure Data Lake (both Storage & Analytics) has been in public preview for a month or two.
You can get started by reading this.
I thought I would kick some posts about more complex scenarios...
-
Azure Data Lake Analytics Quick Start
Solution ·UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.
Azure Data Lake (both Storage & Analytics) has been in public preview for a month or two.
It already has surprisingly good documentation:
- Overview of U-SQL ; walks you through diverse scenarios, ramping you...
-
Checkpoints in ETL
Solution ·UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.
Extract, Transform & Load (ETL) are so 2000’s. But then again, they are more popular than ever!
On one hand, ETL are from an era where you didn’t care too much about real-time insights or couldn’t afford it because of technical limitation. ...
-
HDInsight Hadoop Hive - CSV files analysis
Solution ·![hive_logo_medium[1]](/assets/posts/2015/4/hdinsight-hadoop-hive-csv-files-analysis/hive_logo_medium1.jpg)
Ok, on a past blog we’ve been setuping Azure HDInsight for some Hive fun.So let’s!
Today I’ll go and analyse the data contained in multiple CSV files. Those files will be created (in Excel) but in a real-world scenario, they could be either data dump on a file server...
-
Azure Data Lake - Early look
Solution ·UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.
Ok, this is a super early look at the technology. Azure Data Lake was announced yesterday (September 29th, 2015) at AzureCon (and later blogged about by Scott Gu), it will public preview at the end of...
-
HDInsight Hadoop Hive - Setup
Solution ·![hive_logo_medium[1]](/assets/posts/2015/3/hdinsight-hadoop-hive-setup/hive_logo_medium1.jpg)
Let’s explore Hadoop Hive, shall we?It was originally build by Facebook as an abstraction on top of Hadoop Map Reduce and now is an open source...
-
Hadoop ecosystem overview
Solution ·
Have taken a look at Hadoop lately?People who do not know Hadoop think it’s a big data technology the same way SQL Server is a technology.
But Hadoop is more of an ecosystem of different modules interacting together. This is its major strength and also the source of its major...
![pexels-photo-756901[1]](/assets/posts/2018/1/azure-databricks-rdd-resilient-distributed-dataset/pexels-photo-7569011.jpg "pexels-photo-756901[1]")
![Databricks_logo[1]](/assets/posts/2017/4/azure-databricks-getting-started/databricks_logo1.png "Databricks_logo[1]"){kind=logo}
BigData
-
Azure Data Explorer (Kusto)
Solution ·
Let’s talk about Azure Data Explorer (ADX
) also known as Kusto.If you ask me that is the best kept secret in Azure.
Well, it isn’t exactly a secret but most people do not know about it or if they do, they just think of it as the back-end...
Business
-
Digital Transformation - Investing in the Core
Solution ·
Digital Transformation is all the rage. It’s a little like virtue: everybody is rooting for it but few can actually articulate what it is!I recently found some nice theoretical framework which simplifies its explanation.
We will first refer to that theoretical framework. We will then articulate how it...
Containers
-
Ingress rules in different Kubernetes namespaces
Solution ·
In this article I want to show how an ingress controller in Kubernetes can be used to route traffic to workloads deployed in multiple namespaces.
The online doc for AKS deploys everything in the same namespace. Hence this article is a thin extension to the online doc.
The basic trick is to deploy...
-
Monitoring metrics in AKS
Solution ·
AKS has a nice integration with Azure monitor. Out of the box there are a couple of dashboards for common metrics.
What if you need to go beyond those metrics?
This is what we’re going to do in this article. I’m going to show how to get the CPU usage per container. Along...
-
Requests vs Limits in Kubernetes
Solution ·
Kubernetes doesn’t know what resources (i.e. CPU & memory) your container needs. That is why you must give it some hints.
If you run way under capacity and / or fairly similar pods, you do not need to do that. But if you start approaching the maximum capacity of your cluster or if you...
-
Testing outbound connections in AKS-Kubenet and ACI
Solution ·
What happens if a pod in AKS initiates a connection with a private endpoint? Which private IP address does the outbound connection uses?
This is relevant for a private IP inside the same VNET, a peered VNET or an IP accessible via a VPN or Express Route.
In general, the private IP of the...
-
AKS Auto Scaler with ARM Template
Solution ·
Azure Kubernetes Service (AKS) Auto Scaler is finally out there in public preview!
The online documentation does a great job of getting us started. In this article I wanted to get it a little further with two things. First by showing how to use ARM templates to deploy an AKS Cluster with...
-
Network Policies with Calico in AKS
Solution ·
Calico network plugin is finally supported within Azure Kubernetes Services (AKS).
There is a very good tutorial on the online documentation, so we won’t give a walkthrough here.
Instead we will highlight a couple of points about Network Policies in general and in AKS.
Online references
On top of the aforementioned AKS...
-
Flex Volume in AKS
Solution ·
I wanted to start looking at a few modules helping integrate AKS with the rest of Azure.
In a past article, we looked at Pod Identity.
This time around, we will look at Flex Volume.
Flex Volume is an integration between AKS and Key Vault. It allows keys, secrets & certificates (the...
-
Azure AD Pod Identity in AKS
Solution ·
I wanted to start looking at a few modules helping integrate AKS with the rest of Azure.
A big integration point is identity. For many reasons, we’ll want our pods to use service principal identities:
- Access an Azure service supporting AAD-integration
-
Custom Logs on AKS & Azure Monitor
Solution ·
Let’s look at a concrete problem:
- I have containers deployed in AKS
- Those container log into custom files
- I want to analyse those logs using Azure Monitor (Log Analytics)
We’ll look at how to do that.
We leverage Azure Monitor for containers.
Custom logs in Log Analytics also is...
-
Dynamic disks in AKS
Solution ·
Some workloads running on Azure Kubernetes Service (AKS) requires persisting state on disk.
In general, I recommend to use external PaaS services, i.e. Azure Blob Storage, Azure SQL DB, Azure Cosmos DB, etc. . Those services take care of the stateful nature of the service, manages HA, backups, geo-replication, etc. .
Persisting state...
-
Designing Distributed Systems by Brendan Burns
Solution ·
In this article I wanted to talk about a book I just finished reading.
Although its title and scope is around distributed systems in general, Brendan Burns discussion is mostly using containers & Kubernetes.
Brendan Burns is currently (as of this writing in November 2018) at Microsoft. He is a cofounder of...
-
Understanding multiple Ingress in AKS
Solution ·
Last time, I covered the ins and outs of Ingress on AKS.
We looked at how to install nginx ingress controller and how the controller is deployed as a load balanced service. We did some URL based routing and domain name overloading.
This was in the spirit of clarifying the magic...
-
Understanding simple HTTP Ingress in AKS
Solution ·
We looked at Kubernetes Ingress conceptually. We looked at different use cases: URL based routing and multiple domains.
We also looked at how ingress was implemented from an AKS perspective, i.e. where traffic was routed in nodes.
In this article, I wanted to get hands on. I figured we could start...
-
Authoring a Helm Chart on Kubernetes / AKS - Parametrized values
Solution ·
In the last article, we authored a very simple chart.
In this article I want to show how to use one of the powerful features of Helm: values.
Values act as parameters to a chart.
Having parameters is key for dev ops as we want to deploy the same chart to different...
-
Authoring a Helm Chart on Kubernetes / AKS - Getting started
Solution ·
As we discussed in our last article, Helm is a great technology. I recommend using it not only to manage third party packages but also your own custom-developed solutions.
But how do you author a Helm Chart?
I found that question hard to answer. The common answer is “look at existing ones”....
-
5 reasons to use Helm charts in Kubernetes
Solution ·
Kubernetes is a big platform to absorb. It has a rich resource & networking model. In many ways it is a miniature version of a fully fledge cloud provider, abstracting the underlying cloud provider.
That is my excuse for postponing addressing Helm for such a long time. Until today.
I’ll look...
-
Azure Dev Ops CI / CD pipelines with AKS - Lessons learned
Solution ·I have this personal project where I have a swag of micro services collaborating to serve different web applications.
I started to work on the Continuous Integration / Continuous Deployment (CI / CD) using Azure DevOps.
I did toy around those areas before. But this was really a rubber hits the road experience...
-
About Ingress in Azure Kubernetes Service (AKS)
Solution ·I did a bit of experimentation with Kubernetes Ingress, more specifically NGINX, lately.
I found the concept of Ingress utterly confusing at first. It is actually relatively simple. So, I thought I would share this sense of simplicity.
This article is conceptual. There will be no code nor even Portal tour.
If you want to ramp up on Ingress in...
-
AKS with Kubenet vs Azure Networking plug-in
Solution ·Update (22-03-2019): This configuration is now officially documented.
I’ve been diving into Kubernetes / AKS Networking lately. I thought I would share some of the insights I stumble upon.
We know AKS has two types of networking, basic & advanced, right?
- Basic provisions its own VNET and exposes only public IPs ...
-
Deploying AKS with ARM Template – Network integration
Solution ·In a past article, we looked at how Azure Kubernetes Services (AKS) integrated with Azure Networking.
AKS is a managed Kubernetes service in Azure.
In this article, we are going to do two things:
- Deploy an AKS cluster with Advanced Networking using an Azure ARM Template.
- Deploy a service on the...
-
Kubernetes Services in Azure AKS - Network integration
Solution ·
Azure Kubernetes Services (AKS) is a managed Kubernetes service in Azure. Not only does it provides a managed cluster, it also integrates with Azure services.
In this article we’ll explain the integration between Kubernetes’ services and Azure Networking. We won’t give code sample though as we’ll keep that for a future article.
…
-
Docker multi-stage build
Solution ·
I recently came across a really nice feature of Docker Build and wanted to share it.Building a Docker Image often requires to build binaries. In my case I needed to build a .NET core application. You might need to build a Java App or any other types of...
-
Cosmos DB Configuration Management
Solution ·I often found the need to automate Cosmos DB deployments.I like to have users run an ARM template and be ready to go. Automation is key component to repeatability. It enables automated testing and increase agility in many other ways.
ARM Templates don’t go beyond a database account.
…
-
Using Azure Container Instances to complete automation
Solution ·We looked at Azure Container Instances.It allows us to launch a container without a cluster and without a VM.
It is ideal for bursting scenarios.
In this article we’ll look at another scenario for container instances: automation.
The limits of ARM Templates
A great tool for automation...
-
More reliable Replica Sets in AKS - Part 2
Solution ·
In Part 1, we discussed how, by default, replica sets in AKS aren’t highly available. In this article, we conclude the discussion. We propose an approach to deploy highly available replica sets.The objective of this article isn’t to propose a novel way to deploy replica sets in...
-
More reliable Replica Sets in AKS - Part 1
Solution ·
Availability is a core architecture attribute often sought after.We’ve taken a look at Azure Managed Kubernetes Cluster (AKS) here. In this article (part 1), we’re going to experiment and prove that replica set aren’t “highly available” by default. In part 2 , we’re going to look at how...
-
Understanding identities in Azure AKS / Kubernetes
Solution ·
We’ve recently looked at Azure AKS (Kubernetes Cluster Managed Services). We’ve looked at how to create a Kubernetes Cluster with 3 lines of Azure CLI.With this we are able to interact with the cluster & deploy containers.
There is a bit of “auto magic” happening in...
-
Get Started with Kubernetes (AKS) in Azure
Solution ·
A few weeks ago we wrote an article about how to get started with Docker in Azure.This time we’ll do it with Kubernetes using Azure AKS Service.
What is Kubernetes?
From https://kubernetes.io/:
Kubernetes is an open-source system for automating deployment, scaling, and management of...
-
Azure Container Registry - Getting Started
Solution ·
When we looked at an overview of Container Services in Azure, we took a look at Azure Container Registry (ACR).A Docker Registry holds Docker images in repositories. Any agent can build & push those images. Nodes running Docker containers can then pull those images.
The main...
-
Azure Container Instance - Getting Started
Solution ·
We’ve looked at an overview of Docker Containers in Azure. One of the services we talked about is Azure Container Instance (ACI).ACI allows us to start a Docker Container without a cluster, without even a VM.
It is ideal for bursting, recurrent jobs and dev-test.
We...
-
Getting started with Docker Containers in Azure
Solution ·
Let’s get started with Docker on Azure.We’ll play a little with Docker Containers. In our container overview we’ve done a tour of the different container managed services.
For a first hands on trial, we’ll go a little more low level and use Docker on a VM. We’ll...
-
Linux Custom Script - Docker Sandbox
Solution ·I do a lot of proof of concepts (POCs) as part of my job.I hate keeping demo environment around. They tend to become brittle, out-of-date and filled with the last stuff I did.
I prefer to start from a clean slate every single time when possible.
That means...
-
Overview of Docker Containers in Azure
Solution ·
Let’s talk about Containers in Azure.
In this article, we’ll cover services related to Containers in Azure. We’ll see what scenarios they each address.
Azure focuses on Docker for container. There are other container technologies out there. But Docker is by far the most popular.
Our...
-
Docker Containers on Windows Server
Solution ·If you had any doubts about the increased pace in IT innovation, look at Docker Containers. The project was open sources in March 2013 as a container technology for Linux and 1.5 years later, in Octobre 2014, Microsoft announced they were integrating that technology on Windows Server 2016!
That’s 1.5 years from toe in the water to major...
Data
-
Impersonating user in ADLS with Kusto
Solution ·
We discussed Azure Data Explorer (ADX) and its query language Kusto in a past article.
In this article I wanted to show how to access an ADLS gen 2 using user impersonation, i.e. using the identity of the user running the queries to access the storage.
It would be all trivial and boring...
-
Azure Data Explorer (Kusto)
Solution ·Let’s talk about Azure Data Explorer (ADX
) also known as Kusto.If you ask me that is the best kept secret in Azure.
Well, it isn’t exactly a secret but most people do not know about it or if they do, they just think of it as the back-end...
-
How to use Azure Data Lake Storage REST API
Solution ·Azure Data Lake Storage (ADLS) Generation 2 has been around for a few months now.
That new generation of Azure Data Lake Storage integrates with Azure Storage. This makes it a service available in every Azure region. It also makes it easier to access as it is built on foundation well known...
-
Monitoring metrics in AKS
Solution ·AKS has a nice integration with Azure monitor. Out of the box there are a couple of dashboards for common metrics.
What if you need to go beyond those metrics?
This is what we’re going to do in this article. I’m going to show how to get the CPU usage per container. Along...
-
Creating a Backup Solution for Cosmos DB using change feed
Solution ·
Cosmos DB is Azure native NoSQL database. It has awesome capabilities such as global-distribution, exceptionally high availability, throughput scalability, and much, much more.
As with most NoSQL horizontally scalable databases it doesn’t have the same backup capabilities as mainstream RDBMS systems.
Cosmos DB has an automated backup...
-
Cosmos DB Stored Procedures - handling continuation
Solution ·
I’ve recently did some work involving Stored Procedures in Cosmos DB.There are a few techniques to learn when our stored procedures handle large data sets. It is all about continuation.
In this article, we’ll look at how to build a robust and scalable stored procedure.
We’ll start with...
-
Event Hubs ingestion performance and throughput
Solution ·
Azure Event Hubs is a data streaming platform as a Service. It is an ingestion service.We’ve looked at Event Hubs as the ingestion end of Azure Stream Analytics in two recent articles (here & here).
Here we look at client side performance of different...
-
Implementing & Automating Azure Stream Analytics Pipeline
Solution ·In our last article, we set out to build a resilient architecture around streaming events.In this article, we are going to build the solution. We are going to use an ARM template which automates deployment. We will also go through the configuration of different services.
The goal...
-
Taming the fire hose: Azure Stream Analytics
Solution ·
We came upon an interesting challenge recently.Let’s have a stream of events pouring in. The source is unimportant. Hundreds, peeking at thousands, of events per second.
We need to update a table in Azure SQL with a summary of the events.
Let’s just assume the events are about...
-
Python Version in Databricks
Solution ·In the last few months, we’ve looked at Azure Databricks:
- Getting Started
- Resilient Distributed Dataset
- Spark SQL – Data Frames
- Transforming Data Frames in Spark
- Parsing escaping CSV files in Spark
- Import Notebooks in Databricks
In those articles, we used the Python SDK (also... -
Import Notebooks in Databricks
Solution ·We’ve recently looked Azure Databricks:
- Getting Started
- Resilient Distributed Dataset
- Spark SQL – Data Frames
- Transforming Data Frames in Spark
- Parsing escaping CSV files in Spark
In most cases we did share notebooks on GitHub.
Here we wanted to show...
-
Azure Databricks - Parsing escaping CSV files in Spark
Solution ·In previous weeks, we’ve looked at Azure Databricks, Azure’s managed Spark cluster service.We then looked at Resilient Distributed Datasets (RDDs) & Spark SQL / Data Frames. We also looked at an example of more tedious transformation prior to querying using the H-1B Visa...
-
Azure Databricks - Transforming Data Frames in Spark
Solution ·In previous weeks, we’ve looked at Azure Databricks, Azure’s managed Spark cluster service.We then looked at Resilient Distributed Datasets (RDDs) & Spark SQL / Data Frames.
We wanted to look at some more Data Frames, with a bigger data set, more precisely some...
-
Azure Databricks - Spark SQL - Data Frames
Solution ·We looked at Azure Databricks a few weeks ago.Azure Databricks is a managed Apache Spark Cluster service.
More recently we looked at how to analyze a data set using Resilient Distributed Dataset (RDD). We used the Social characteristics of the Marvel Universe public dataset,...
-
Azure Databricks - RDD - Resilient Distributed Dataset
Solution ·We looked at Azure Databricks a few weeks ago. Azure Databricks is a managed Apache Spark Cluster service.In this article, we are going to look at & use a fundamental building block of Apache Spark: Resilient Distributed Dataset or RDD. We are going to use the...
-
Azure Databricks - Getting Started
Solution ·Apache Spark is rising in popularity as a Big Data platform. It exists on this accelerated timeline for such an impactful technology.Think about it:
- 2009, started as a Berkeley’s University project.
- 2010, open sourced
- 2013, donated to Apache Foundation
- 2014, becomes...
-
Disaster Recovery with VM Scale Sets & Geo-Replicated DBs
Solution ·![train-wreck-steam-locomotive-locomotive-railway-73821[1]](/assets/posts/2017/4/disaster-recovery-with-vm-scale-sets-geo-replicated-dbs/train-wreck-steam-locomotive-locomotive-railway-738211_thumb.jpg "train-wreck-steam-locomotive-locomotive-railway-73821[1]")
Last year we posted an article about different options available in Azure to implement a disaster recovery strategy.We strongly suggest to review that article as it gives good insights about what a disaster recovery strategy is within an already resilient Cloud Environment but also clear out a...
-
Setup for populating Cosmos DB with random data using Logic Apps
Solution ·We recently published an article about Cosmos DB Performance with Geospatial Data.In this article, we’re going to explain how to setup the environment in order to run those performance test.
More importantly, we believe this article is interesting on its own as it shows how to use...
-
Cosmos DB Performance with Geospatial Data
Solution ·![pexels-photo-348489[1]](/assets/posts/2017/4/cosmos-db-performance-with-geospatial-data/pexels-photo-3484891_thumb.jpg "pexels-photo-348489[1]")
Time for some performance tests on Cosmos DB around Geospatial coordinates!Let’s hurt the service and see where it shines, where it has a hard time and how scaling it (i.e. increasing Request Units or RUs) helps.
We’ll publish an how-to create the same setup in a future article...
-
Hacking: accessing a graph in Cosmos DB with SQL / DocumentDB API
Solution ·![pexels-photo-264635[1]](/assets/posts/2017/3/hacking-accessing-a-graph-in-cosmos-db-with-sql-documentdb-api/pexels-photo-2646351_thumb.jpg "pexels-photo-264635[1]")
Azure Cosmos DB is Microsoft’s globally distributed multi-model database service.At this point in time (August 2017) there are four supported models: DocumentDB (also named SQL because the query language is similar to T-SQL), MongoDB, Tabular & Gremlin.
-
Cosmos DB Async Querying & Streaming
Solution ·![pexels-photo-223022[1]](/assets/posts/2017/3/cosmos-db-async-querying-streaming/pexels-photo-2230221.jpg "pexels-photo-223022[1]")
I wrote an article back in January 2015 about async querying Azure DocumentDB using the .NET SDK.The service was still in preview back then.
Since then DocumentDB has been superseded by Azure Cosmos DB and the SDK has changed a bit so I thought I would...
-
Hyperspheres & the curse of dimensionality
Solution ·
I previously talked about the curse of dimensionality (more than 2 years ago) related to Machine Learning.Here I wanted to discuss it in more depth and dive into the mathematics of it.
High dimensions might sound like Physics’ string theory where our...
-
Extended Outage @ Instapaper - Resiliency example
Solution ·![art-broken-explosion-glass[1]](/assets/posts/2017/1/extended-outage-instapaper-resiliency-example/art-broken-explosion-glass1_thumb.jpg "art-broken-explosion-glass[1]")
I use Instapaper extensively to store the continuous flow of internet articles I want to read. I created a bunch of tools integrating with it (e.g. monitoring atom feeds and sending new articles to Instapaper)Last week my tools didn’t work for a while so I finally logged...
-
Azure SQL Elastic Pool – Moving databases across pools using PowerShell
Solution ·I’ve written a bit about Azure SQL Elastic Pool lately: an overview, about ARM template and about database size.
One of the many great features of Azure SQL Elastic Pool is that like Azure SQL Database (standalone), we can change the eDTU capacity of the pool...
-
Azure SQL Elastic Pool – Database Size
Solution ·I mentioned in a past article, regarding database sizes within an elastic pool:“No policies limit an individual database to take more storage although a database maximum size can be set on a per-database basis.”
I’m going to focus on that in this article.
An Azure SQL Database resource has...
-
Azure SQL Elastic Pool - ARM Templates
Solution ·In my last article, I covered Azure SQL Elastic Pool. In this one I cover how to provision it using ARM templates.As of today (December 2016), the documentation about Azure SQL Elastic Pool provisioning via ARM templates is… not existing.
Searching for it I was able...
-
Azure SQL Elastic Pool Overview
Solution ·What is
![rubber-bands-1158199_640[1]](/assets/posts/2016/4/azure-sql-elastic-pool-overview/rubber-bands-1158199_6401_thumb.jpg "rubber-bands-1158199_640[1]")
Azure SQL Elastic Pool and what it brings to Azure SQL Database, the SQL Azure Platform as a Service (PaaS).Traditional model
Let’s look at how Azure SQL works without elastic pools first.
Azure SQL Database comes with...
-
Primer on Azure Monitor
Solution ·Azure Monitor is the latest evolution of a set of technologies allowing Azure resources monitoring.
I’ve written about going the extra mile to be able to analyze logs in the past.
The thing is that once our stuff is in production with tons of users hitting...
-
How does Azure Data Warehouse scale?
Solution ·I’ve been diving in the fantastical world of Azure Data Warehouse (ADW) in the last couple of days.
I’ve been reading through all the documentation on Azure.com. If you are serious about mastering that service I advise you do the same: it is a worthy read.
In this article, I wanted to summarize a few concepts...
-
DocumentDB protocol support for MongoDB
Solution ·
Microsoft announced, in the wake of many DocumentDB announcement, that DocumentDB would support MongoDB protocol.What does that mean?
It means you can now swap a DocumentDB for a MongoDB and the client (e.g. your web application) will work the same.
This is huge.
It...
-
How to do Data Science
Solution ·
These days, it’s all about Data Science.
What is Data Science?
Last month Brandon Rohrer, from the Cortana Intelligence and Machine Learning Blog, came up with an excellent post.
The post basically goes over the workflow I reproduced at the right here.
I found...
-
What is Statistics and why should you care?
Solution ·
Unless you graduated in art, chances are you did a course in Statistics.Chances are you hated it.
Most people I know postponed that course until the end of their degree, didn’t understand much about it and hated it dearly.
I didn’t like it either and understood very little.
A few...
-
Power BI Embedded in Azure
Solution ·One of the many announces of Build 2016 (last week) is Power BI Embedded.
On the surface you might wonder what’s the difference between that and Power BI Service itself?
Power BI Service is a powerful tool. It allows you to author reports & dashboard in Power BI desktop and export them in Power BI...
-
How to copy and backup SQL Databases in Azure
Solution ·Let’s say you want a copy of your database for some reasons. Scenarios might vary. Maybe you want to bring back production DB to another environment.
There are at least three ways I could think to do that with the Azure Portal:
- Copy the Database
- Restore a version of the Database
- Backup / Restore the current Database
Copy...
-
Row Level Security in 3-tiers applications
Solution ·Row-Level Security is a great new feature in SQL. It is already available in Azure SQL Database and will be available in SQL Server 2016.
This is Cloud-First for you.
Row-Level Security allows you to forbid a user to see and/or access data they shouldn’t be able to see. Traditionally this is implemented at the application level, for...
-
Stategic Data, Infonomics & Big Data
Solution, Strategy ·UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.
I wanted to talk a bit about your Strategic Data & the concept of Data Lake (regardless of its implementation).Nowaday, data is seen less and less as...
-
Checkpoints in ETL
Solution ·UPDATE (19-01-2016): Have a look at Azure Data Lake series for more posts on Azure Data Lake.
Extract, Transform & Load (ETL) are so 2000’s. But then again, they are more popular than ever!
On one hand, ETL are from an era where you didn’t care too much about real-time insights or couldn’t afford it because of technical limitation. ...
-
Azure Key Vault & SQL Server Connector Update
Solution ·![enhance-data-protection[1]](/assets/posts/2015/4/azure-key-vault-sql-server-connector-update/enhance-data-protection1.png)
Azure Key Vault is alive and well!The Azure service allowing you to store keys and secrets in a secured container has been released at the end of summer and it continues to improve.
The SQL Server Connector is a component that can be installed on SQL Server...
-
HDInsight Hadoop Hive - CSV files analysis
Solution ·Ok, on a past blog we’ve been setuping Azure HDInsight for some Hive fun.So let’s!
Today I’ll go and analyse the data contained in multiple CSV files. Those files will be created (in Excel) but in a real-world scenario, they could be either data dump on a file server...
-
HDInsight Hadoop Hive - Setup
Solution ·Let’s explore Hadoop Hive, shall we?It was originally build by Facebook as an abstraction on top of Hadoop Map Reduce and now is an open source...
-
Hadoop ecosystem overview
Solution ·Have taken a look at Hadoop lately?People who do not know Hadoop think it’s a big data technology the same way SQL Server is a technology.
But Hadoop is more of an ecosystem of different modules interacting together. This is its major strength and also the source of its major...
-
SQL Server In-Memory value and use cases
Solution ·![EPROM02[1]](/assets/posts/2015/3/sql-server-in-memory-value-and-use-cases/eprom021.png?w=300)
Microsoft SQL Server 2014 has made it in the leader category of Forrester’s recent In-Memory Database Platforms, Q3 2015 report.
SQL Server 2014 was behind SAP’s Hana, Oracle TimesTen & IBM DB2 with BLU acceleration. It was on pare with Teradata Intelligent Memory. Those were the leaders. Following were...
-
Analysing Application Logs with DocumentDb
Solution ·![IC791289[1]](/assets/posts/2015/3/analysing-application-logs-with-documentdb/ic7912891.png?w=300)
Azure DocumentDB is Microsoft Document-centric NoSQL offering in the cloud.I’ve been working with it since Septembre 2014 and I wanted to share a use case I found it really good at: log analysis.
Now that takes some context.
I have been working for a customer using Microsoft Azure...
-
Azure SQL Datawarehouse
Solution ·Documentation on Azure SQL Datawarehouse, the new Azure Datawarehouse managed service, is quite thin.
The online documentation, as of today (24/07/2015), consists of 3 videos and a blog post.
Here is what I gathered.
![db[1]](/assets/posts/2015/3/azure-sql-datawarehouse/db1_thumb.png "db[1]")
One of the great characteristic of the offering is the separate Storage...
-
SQL Server 2016
Solution ·Here’s a rundown of my favourite new features in SQL Server 2016, largely inspired by the SQL Server Evolution video.
Impact of Cloud-First on SQL Design
This is a really nice illustration of the consequences of Cloud-First for Microsoft products.
![4150.Microsoft-Mobile-First-Cloud-First[1]](/assets/posts/2015/3/sql-server-2016/4150-microsoft-mobile-first-cloud-first1_thumb.jpg "4150.Microsoft-Mobile-First-Cloud-First[1]")
SQL has been basically flipped around as a product. ...
-
Machine Learning - An Introduction - Part 2
Solution ·In a past blog entry I gave an overview of what Machine Learning is. I showed a simple linear regression example. The goal really was to explain to newcomer to the field what Machine Learning is, what type of problem it tries to solve and what the general approach is.
Of course, I used an extremely simple example in...
-
Machine Learning - An introduction - Part 1
Solution ·As I mentioned before, I did specialize (through graduated studies) in Machine Learning only to drop the field after a few years of trial on the Marketplace. I felt the field wasn’t ready for prime industrial applications.
Years have past, the field has matured and now is an exciting time to...
-
Azure Data Factory Editor (ADF Editor)
Solution ·Azure Data Factory is still in preview but obviously has a committed team behind it.
![networking[1]](/assets/posts/2015/2/azure-data-factory-editor-adf-editor/networking1.jpg?w=300)
When I looked at the Service when the preview was made available in last Novembre, the first thing that stroke me was the lack of editor, of designing surface. Instead, you had to configure...
-
Azure DocumentDB: first use cases
Solution ·A few weeks ago Microsoft released (in preview mode) its new NoSQL Database: DocumentDB.
Not Only SQL (NoSQL) databases are typically segmented in the following categories: Key-Value (e.g. Azure Table Storage, Redis), Column (e.g. HBase, Cassandra), Document (e.g. CouchDB, MongoDB) & Graph. By its name but mostly by its feature set, DocumentDB falls in the document category.
…
-
Entity Framework with Asynchronous behaviours
Solution ·They finally did it: the future release of Entity Framework (version 6) will sport asynchronous behaviour based on .NET 4.0 Task Parallel Library (TPL).
The API is pretty neat. First the SaveChanges gets an async brother SaveChangesAsync returning a Task. So we can now write things like:
await context.SavesChangesAsync();
The more complicated topic is the queries. ...
-
Windows Azure SQL Database named an Enterprise Cloud Database Leader by Forrester Research
Solution ·Three weeks ago Forrester released a paper on Cloud Databases.
As pointed out by Microsoft, Forrester declared SQL Azure and Amazon Relational Database Service (RDS) and Amazon DynamoDB and salesforce.com’s Database.com as leader of the pack.
That is quite impressive given the relatively late start Microsoft took on those competitors.
SQL Azure is a leading service in the...
-
Delaying action at transaction commit
Solution ·I am designing a back-end where Database contention is the number-one issue for scalability.
We use those darn distributed transactions and therefore I do not know how long the transaction my component is participating into will last. If a transaction remains open for a few seconds, many of those would create contention in the Database my component is using. This...
-
A Hitchhiker's Guide to StreamInsight 2.1 Queries
Solution ·Microsoft has released a very interesting tutorial (in PDF format) with an accompanying VS Solution.
-
StreamInsight in the Cloud: Meet Project Codename “Austin”
Solution ·Last week I’ve talked about Microsoft StreamInsight, an engine to do real time data analysis and a nice article overviewing at just the depth (for me anyway).
Well, StreamInsight might one day even be available in the cloud with Project Codename “Austin”.
Already available as a private CTP, this product will be a nice addition to...
-
Microsoft StreamInsight Intro
Solution ·Microsoft released Microsoft StreamInsight with SQL Server 2008 R2.
This is an engine for real time data analysis. It’s an alternative to traditional BI which typically crunch replicated data. Instead, StreamInsight let’s you analyse the data as it comes to you (as a stream).
The programming model is built on top of LINQ with the twist that the...
-
SQL Azure & ACID Transactions: not so much in back to 2001
Solution ·I wrote a blog entry back in November 2010 about ACID Transactions & SQL Azure.
I assumed light transactions using System.Transactions.TransactionScope were not supported based on an article I’ve just read back then. The article suggested using explicit transaction management (i.e. BeginTransaction, Commit & Rollback on the returned SqlTransaction object) which I found quite...
-
SQL Azure Federation: Automating your database partitioning (sharding)
Solution ·Microsoft is about to introduce the federation concept into SQL Azure. You can read about it on Microsoft SQL Azure Program Manager Cihan Biyikoglu’s blog:
-
7 Things You Need To Know about SQL Azure Reporting
Solution ·
The SQL Azure Team had a quite interesting blog back in mid-November around the newly released SQL Azure Reporting. Here’s a summary of their nuggets about the new technology.- It’s Based on SQL Server Reporting Services (SSRS)
SQL Azure Reporting provides a subset of the features of...
- It’s Based on SQL Server Reporting Services (SSRS)
-
SQL Azure & ACID Transactions: back to 2001
Solution ·I meant to write about this since I read about it a little back in July, today is the day.
You know I love Microsoft SQL Azure.The technology impressed me when it was released. Until then Azure contained only Azure storage. Azure Storage is a...
-
Forrester: SQL Azure Raises The Bar On Cloud Databases
Solution ·
November 2nd 2010, Forester Research published a report around Microsoft SQL Azure. The report can be found on Microsoft web site:http://www.microsoft.com/presspass/itanalyst/docs/11-02-10SQLAzure.PDF
Basically, they interviewed 26 companies using the technology and concluded that:
- SQL Azure is reliable
- It delivers for small to medium scenarios ...
-
Querying Azure Storage
Solution ·I’ve been reading the Wikipedia article about CouchDB lately. It always impresses me how much engineering effort and men-hours are put into so many competing platforms. I guess the least we could do is to look at the architecture of those initiatives and get the best out of them.
<img style="display:inline;margin-left:0;margin-right:0;" alt="CouchDB - Relax" align="right" src="http://couchdb.apache.org/img/couchdb-logo.png" width="400" height="110"...
-
Robust Non-ACID Transaction pattern in Azure
Solution ·I’ve been working some more on some detailed design of the Azure Pub-Sub CodePlex project. This project aims at creating a pub-sub messaging infrastructure running on Windows Azure & using Azure Storage.
Azure storage doesn’t support ACID transactions in general. Each operation is a transaction within a given partition, but you can’t span a transaction on many operations. Now...
![train-wreck-steam-locomotive-locomotive-railway-73821[1]](/assets/posts/2017/4/disaster-recovery-with-vm-scale-sets-geo-replicated-dbs/train-wreck-steam-locomotive-locomotive-railway-738211.jpg "train-wreck-steam-locomotive-locomotive-railway-73821[1]")
![pexels-photo-348489[1]](/assets/posts/2017/4/cosmos-db-performance-with-geospatial-data/pexels-photo-3484891.jpg "pexels-photo-348489[1]")
![pexels-photo-264635[1]](/assets/posts/2017/3/hacking-accessing-a-graph-in-cosmos-db-with-sql-documentdb-api/pexels-photo-2646351.jpg "pexels-photo-264635[1]")
![art-broken-explosion-glass[1]](/assets/posts/2017/1/extended-outage-instapaper-resiliency-example/art-broken-explosion-glass1.jpg "art-broken-explosion-glass[1]")
![rubber-bands-1158199_640[1]](/assets/posts/2016/4/azure-sql-elastic-pool-overview/rubber-bands-1158199_6401.jpg "rubber-bands-1158199_640[1]")
![EPROM02[1]](/assets/posts/2015/3/sql-server-in-memory-value-and-use-cases/eprom021.png)
![IC791289[1]](/assets/posts/2015/3/analysing-application-logs-with-documentdb/ic7912891.png)
![db[1]](/assets/posts/2015/3/azure-sql-datawarehouse/db1.png "db[1]")
![4150.Microsoft-Mobile-First-Cloud-First[1]](/assets/posts/2015/3/sql-server-2016/4150-microsoft-mobile-first-cloud-first1.jpg "4150.Microsoft-Mobile-First-Cloud-First[1]")
DevOps
-
Deploying an ARM template hierarchy
Solution ·ARM templates are a powerful way to deploy Azure resources.
It’s a declarative language which express the target state. It can therefore be used to create or update resources.
An ARM template is a JSON file. Once a target is big enough it becomes convenient to split the ARM template into multiple files....
-
Azure Application Insights Role Name in web page
Solution ·
We discussed using Role Name to separate telemetries from different applications in a previous article.
We discuss the Nuget package I developed to set the role name for each telemetry in a backend application.
In this article I wanted to cover another element that receives little coverage: how to do that for...
-
Azure Application Insights Role Name Nuget Package
Solution ·
I’ve been using Azure Application Insights to monitor micro-services.
Since I have many micro-services, it isn’t practical to have an Application Insight resource per service. Given that those micro-services interact with each other and I want to monitor those interactions, it makes sense to get all telemetry in one place.
The issue with...
-
Using Azure DevOps REST API to start multiple releases with Azure Logic Apps
Solution ·In our last article, we looked at how we could leverage the Azure DevOps REST API to trigger multiple releases.
This is useful when rebuilding an environment: first we release the shared infrastructure then each service on top of it.
In this article, we’ll see an implementation example using Azure Logic...
-
Using Azure DevOps REST API to start multiple releases
Solution ·Azure DevOps CI / CD (i.e. Build / Release) mechanism is very valuable to me. It makes things reproduceable & robust. It allows me to change whatever I want since I have this constant safety net underneath.
For simple projects, having a build and release is fine. With micro-services I adopt the pattern of...
-
Azure DevOps - Task output to a variable
Solution ·
I’ve been building a couple of Continuous Integration / Continuous Delivery (CI / CD) pipelines on Azure DevOps lately.
Azure DevOps is formerly known as Visual Studio Team Service (VSTS). As I mentioned recently, VSTS had nothing to do with Visual Studio and we’re very happy with the new brand name.
With...
-
Authoring a Helm Chart on Kubernetes / AKS - Parametrized values
Solution ·In the last article, we authored a very simple chart.
In this article I want to show how to use one of the powerful features of Helm: values.
Values act as parameters to a chart.
Having parameters is key for dev ops as we want to deploy the same chart to different...
-
Authoring a Helm Chart on Kubernetes / AKS - Getting started
Solution ·As we discussed in our last article, Helm is a great technology. I recommend using it not only to manage third party packages but also your own custom-developed solutions.
But how do you author a Helm Chart?
I found that question hard to answer. The common answer is “look at existing ones”....
-
5 reasons to use Helm charts in Kubernetes
Solution ·Kubernetes is a big platform to absorb. It has a rich resource & networking model. In many ways it is a miniature version of a fully fledge cloud provider, abstracting the underlying cloud provider.
That is my excuse for postponing addressing Helm for such a long time. Until today.
I’ll look...
-
Azure Dev Ops CI / CD pipelines with AKS - Lessons learned
Solution ·I have this personal project where I have a swag of micro services collaborating to serve different web applications.
I started to work on the Continuous Integration / Continuous Deployment (CI / CD) using Azure DevOps.
I did toy around those areas before. But this was really a rubber hits the road experience...
Identity
-
Network Policies with Calico in AKS
Solution ·Calico network plugin is finally supported within Azure Kubernetes Services (AKS).
There is a very good tutorial on the online documentation, so we won’t give a walkthrough here.
Instead we will highlight a couple of points about Network Policies in general and in AKS.
Online references
On top of the aforementioned AKS...
-
Azure AD Pod Identity in AKS
Solution ·I wanted to start looking at a few modules helping integrate AKS with the rest of Azure.
A big integration point is identity. For many reasons, we’ll want our pods to use service principal identities:
- Access an Azure service supporting AAD-integration
-
Accessing Azure Key Vault using Managed Service Identity Logic Apps
Solution ·Azure Key Vault is a great service to manage secrets, keys & certificates.
It uses RBAC to control access. Like all access control system, there is a chain of access. For instance, my user account has access to the vault: this means if my account’s credentials get leaked, the access to the...
-
Service Principal for Logic App Connector
Solution ·Azure Logic Apps is a powerful integration platform.
It integrates with different services (inside and outside Azure) using connectors.
Connectors are responsible to authenticate to the service they represent.
Some connectors will hold the credentials. This is the case, for instance, of the SQL connector.
Other connectors will by default take the...
-
Creating a Service Principal with Azure CLI
Solution ·Service Principals are a bit of a weird beast. They are Azure Active Directory applicationswith kind of an extra bit. That bit says they can actually login by themselves. Hence the name principal. But being an application is kind of weird.
We covered Service Principals in the past. We covered how to...
-
RBAC and role assignment using ARM Templates
Solution · Azure supports Role Based Access Control (RBAC) as an access control paradigm.It allows to map a user (or a group of users) to a role within a given scope (resource, resource group, subscription or management group).
For instance, we could map my user identity to a Virtual...
-
Understanding identities in Azure AKS / Kubernetes
Solution ·We’ve recently looked at Azure AKS (Kubernetes Cluster Managed Services). We’ve looked at how to create a Kubernetes Cluster with 3 lines of Azure CLI.With this we are able to interact with the cluster & deploy containers.
There is a bit of “auto magic” happening in...
-
Security with API: OAuth, token-based access vs key-based access
Solution ·Let’s consider security with APIs, i.e how to securely identify the caller.There are two authentication methods quite popular in the cloud to secure APIs:
- Key-based access
- OAuth, or token-based access in general
Let’s compare them.
Key-Based
By key-based we mean...
-
Azure Functions HTTP – Authorization Levels
Solution ·In a past article, we looked at Serverless compute in Azure in general and Azure Functions specifically.In this article we wanted to focus on Azure Function triggered by HTTP requests and the different options we have to authenticate:
- Anonymous
- Function
- Admin
- System
- User
Those are...
-
Managing Azure AD Application members in Portal
Solution ·One of Azure AD’s powerful concept is the application. It gives context to an authentication as we explained in this article.
An application can also be used as an authorization barrier since we can manage an application members. This is optional as by default, everyone in a tenant has access to its application. But if we opt in to...
-
Automating Azure AD
Solution ·In the previous article, we explored how to interact (read / write) to an Azure AD tenant using Microsoft Graph API.
In the article before that, we looked at how to authenticate a user without using Azure AD web flow.
Those were motivated by a...
-
Using Microsoft Graph API to interact with Azure AD
Solution ·
In my last article, I showed how to authenticate on Azure AD using a user name / password without using the native web flow.
The underlying scenario was to migrate an application using an LDAP server by leveraging an Azure AD tenant.
The logical continuation of that scenario is...
-
Authenticating to Azure AD non-interactively
Solution ·
I want to use Azure AD as a user directory but I do not want to use its native web authentication mechanism which requires users to go via an Active Directory page to login (which can be branded and customized to look like my own).I just...
-
Joining an ARM Linux VM to AAD Domain Services
Solution ·Active Directory is one of the most popular domain controller / LDAP server around.
In Azure we have Azure Active Directory (AAD). Despite the name, AAD isn’t just a multi-tenant AD. It is built for the cloud.
Sometimes though, it is useful to have a traditional domain controller... in the cloud. Typically this is with legacy workloads built to work...
-
Azure Active Directory Labs Series – Multi-Factor Authentication
Solution ·Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.
In that training there were 8 labs...
-
Azure Active Directory Labs Series – Graph API
Solution ·Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.
In that training there were 8 labs...
-
Azure Active Directory Labs Series – AD Connect
Solution ·Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.
In that training there were 8 labs...
-
Azure Active Directory Labs Series – Creating an AD Forest
Solution ·Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.
In that training there were 8 labs...
-
Azure Active Directory Labs Series – Adding Claims
Solution ·Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.
In that training there were 8...
-
Azure Active Directory Labs Series – Protect Web App
Solution ·
Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.In that training...
-
Azure Active Directory Labs Series – Create Application
Solution ·
Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going to teach.In that training...
-
Azure Active Directory Labs Series - Creating a tenant
Solution ·
Back in June I had the pleasure of delivering a training on Azure Active Directory to two customer crowds. I say pleasure because not only do I love to share knowledge but also, the preparation of the training forces me to go deep on some aspects of what I’m going...
-
Multi-Tenant SaaS with Azure Active Directory B2B & B2C
Solution ·Scenario: I’m creating a Software as a Service (SaaS). I’m having multiple customers & I want to manage their identity. For some of my customers the users won’t have corporate identity ; I would like to offer them to login using their social identity (e.g. Facebook) if they want to or create an account on my site otherwise. For other...
-
Azure Active Directory Application
Solution ·This is a quick post to talk about what an Application is for Azure Active Directory (or Azure AD or AAD).
If you come from the on premise world and are used to Active Directory on Windows Server, the concept of an application within Active Directory probably is a bit foreign.
Think about how authentication works in the...
-
Using Azure Active Directory Service Principal
Solution ·You need an Azure Active Directory (AAD) identity to run some of your services: perhaps an Azure Runbook, Azure SQL Database, etc. .
You could create a normal user in Azure Active Directory and use it. If your AAD is synchronized with an on-premise one, it will get more complicated though. You will need to create it on premise and...
Integration
-
Broadcasting events to multiple solutions
Solution ·
I sat down with a customer who had an interesting integration problem the other day.
They are building a system that creates data useful for the entire enterprise. They want to push that data to different solutions. They also expose APIs, but they want to support the push-model where their system pushes the data...
-
Accessing Azure Key Vault using Managed Service Identity Logic Apps
Solution ·Azure Key Vault is a great service to manage secrets, keys & certificates.
It uses RBAC to control access. Like all access control system, there is a chain of access. For instance, my user account has access to the vault: this means if my account’s credentials get leaked, the access to the...
-
Designing Distributed Systems by Brendan Burns
Solution ·In this article I wanted to talk about a book I just finished reading.
Although its title and scope is around distributed systems in general, Brendan Burns discussion is mostly using containers & Kubernetes.
Brendan Burns is currently (as of this writing in November 2018) at Microsoft. He is a cofounder of...
-
Handling recursivity in Logic Apps
Solution ·
Azure Logic Apps is a powerful integration platform.
Some workflows naturally land themselves to recursion. For instance, handling a hierarchical data structure often has a natural recursive solution.
Azure Logic Apps doesn’t allow recursion directly. It forbids a Logic App invoking itself.
In this article we are going to explore two avenues...
-
Service Principal for Logic App Connector
Solution ·Azure Logic Apps is a powerful integration platform.
It integrates with different services (inside and outside Azure) using connectors.
Connectors are responsible to authenticate to the service they represent.
Some connectors will hold the credentials. This is the case, for instance, of the SQL connector.
Other connectors will by default take the...
-
Implementing & Automating Azure Stream Analytics Pipeline
Solution ·In our last article, we set out to build a resilient architecture around streaming events.In this article, we are going to build the solution. We are going to use an ARM template which automates deployment. We will also go through the configuration of different services.
The goal...
-
Taming the fire hose: Azure Stream Analytics
Solution ·We came upon an interesting challenge recently.Let’s have a stream of events pouring in. The source is unimportant. Hundreds, peeking at thousands, of events per second.
We need to update a table in Azure SQL with a summary of the events.
Let’s just assume the events are about...
-
Setup for populating Cosmos DB with random data using Logic Apps
Solution ·We recently published an article about Cosmos DB Performance with Geospatial Data.In this article, we’re going to explain how to setup the environment in order to run those performance test.
More importantly, we believe this article is interesting on its own as it shows how to use...
-
Invoking a Stored Procedure from a partitioned CosmosDB collection from Logic Apps
Solution ·I struggled a little to make that work, so I thought I would share the learning in order to accelerate your future endeavour.
I was looking at a way to populate a CosmosDB quickly with random data.
Stored Procedures came to mind since they would skip client-server latency. We can call a stored procedure creating hundreds of documents with random...
-
Multi-Tenant SaaS with Azure Active Directory B2B & B2C
Solution ·Scenario: I’m creating a Software as a Service (SaaS). I’m having multiple customers & I want to manage their identity. For some of my customers the users won’t have corporate identity ; I would like to offer them to login using their social identity (e.g. Facebook) if they want to or create an account on my site otherwise. For other...
-
Integration with Azure Service Bus
Solution ·![message[1]](/assets/posts/2015/3/integration-with-azure-service-bus/message1_thumb.png "message[1]")
I’ve been consulting 1.5 years for a customer embarking a journey leveraging Microsoft Azure as an Enterprise platform, helping them rethink their application park.
Characteristic of that customer:
- Lots of Software as a Service (Saas) third parties
- Business is extremely dynamic, in terms of requirements, transitions, partnerships, restructuring,...
-
Azure Data Factory Editor (ADF Editor)
Solution ·Azure Data Factory is still in preview but obviously has a committed team behind it.
When I looked at the Service when the preview was made available in last Novembre, the first thing that stroke me was the lack of editor, of designing surface. Instead, you had to configure...
-
Azure Key Vault
Solution ·Has somebody been peeking on my X-mas list?
Indeed, one of the weakness of the current Azure Paas solution I pointed out last year was that on non-trivial solutions you end up with plenty of secrets (e.g. user-name / password, SAS, account keys, etc.) stored insecurely in your web.config (or similar store).
I was suggesting, as a solution, to...
-
Large Projects
Solution ·There is something about large projects that you'll never find, hence never learn, in smaller projects. The complexity, both technical and in terms of people dynamics, creates an all new set of challenges.
I read the article I Survived an ERP Implementation – Top 10 Gems of Advice I Learned the Hard Way at the beginning of the...
-
Azure ACS fading away
Solution ·ACS is on life support for quite a while now. It was never never fully integrated to the Azure Portal, keeping the UI it had in its Azure Labs day (circa 2010, for those who were born back then).
In an article last summer, Azure Active Directory is the future of ACS, Vittorio Bertocci announces the roadmap: the demise...
-
Securing Azure Messaging Service Bus access
Solution ·I am currently working on a very exciting project involving systems integration across the Azure Messaging Service Bus. I thought I would share some of the painfully acquired knowledge nuggets with you.
About %90 of examples you'll find on Internet uses Azure Bus SDK with 'owner'. That is basically 'admin' privilege because owner...
-
Service Bus EAI and EDI
News ·Back in December (2011), Microsoft released the Azure Service Bus EAI & EDI Labs. This is a sort of CTP with labs.
It’s built on top of AppFabric Service Bus, but seems to contain much more than raw messaging.
According to the MSDN documentation, it seems that a lot of BizTalk concepts were re-used. For instance, look at a...
-
Integration in the Cloud
Solution ·I ran across a series of good blog posts I wanted to share with you, dear readers.
From Richard Seroter, the series is about integration patterns in the cloud, looking at cloud-to-cloud & cloud-to-on-premise scenarios.
Richard looks at three main patterns:
For each...
-
BizTalk beyond 2010
Solution ·I recently blogged about the future of BizTalk and the roadmap of the product version Windows AppFabric.
An even more recent blog post shed some light on the topic.
We are planning to invest in the following main areas:
-
Deep Microsoft Application Platform Alignment
...
-
-
BizTalk vNext: BizTalk Azure?
Solution ·
There are some rumours for the next version of BizTalk (after BizTalk 2010) to be available in the Cloud. A sort of BizTalk Azure. That’s interesting news for sure.
Some are wondering how that will fit with Windows Azure AppFabric. Well, for sure, you’ll need your AppFabric Service Bus in...
![message[1]](/assets/posts/2015/3/integration-with-azure-service-bus/message1.png "message[1]")
Machine Learning
-
Quantum Computing - Scenarios
Solution ·
We recently looked at Quantum Computing value proposition.Quantum Computer can solve some problems by several order of magnitude. This brings today’s intractable problems to be easy to solve tomorrow.
Not all problems have this property. In this article, we’ll look at some problems that do.
This is...
-
Hypersphere Volume
Solution ·
In our last article we looked at how the dimension of data space impacts Machine Learning algorithms. This is often referred to as the curse of dimensionality.At the heart of the article we discussed the fact that an hypersphere hyper-volume trends to zero as dimension increases.
Here...
-
Hyperspheres & the curse of dimensionality
Solution ·I previously talked about the curse of dimensionality (more than 2 years ago) related to Machine Learning.Here I wanted to discuss it in more depth and dive into the mathematics of it.
High dimensions might sound like Physics’ string theory where our...
-
Training a model to predict failures
Solution ·Today a quick entry to talk about a twist on Machine Learning for the predictive maintenance problem.
The Microsoft Cortana Intelligence team wrote an interesting blog the other day: Evaluating Failure Prediction Models for Predictive Maintenance.
When you listen to all the buzz around Machine Learning, it sometimes feels as if we’ve solved all the ML problems and...
-
How to do Data Science
Solution ·These days, it’s all about Data Science.
What is Data Science?
Last month Brandon Rohrer, from the Cortana Intelligence and Machine Learning Blog, came up with an excellent post.
The post basically goes over the workflow I reproduced at the right here.
I found...
-
What is Statistics and why should you care?
Solution ·Unless you graduated in art, chances are you did a course in Statistics.Chances are you hated it.
Most people I know postponed that course until the end of their degree, didn’t understand much about it and hated it dearly.
I didn’t like it either and understood very little.
A few...
-
Where is the statistics in Machine Learning?
Solution ·I often try to explain what Machine Learning is to people outside the field. I’m not always good at it but I am getting better.
One of the confusion I often get when I start to elaborate the details is the presence of statistics in Machine Learning. For people outside the field, statistics are the stuff of survey or their...
-
Strong AI & Existential Risks
Solution ·![0[1]](/assets/posts/2015/3/strong-ai-existential-risks/01.jpg?w=300)
There has been a recrudescence of hysterical talks about Strong Artificial Intelligence (AI) lately.Strong AI is artificial intelligence matching and eventually going beyond the full human cognitive capacity. Weak AI, by opposition, is the replication of some facets of human cognition: face recognition, voice recognition, pattern matching, etc. .
The...
-
Free ebook: Azure Machine Learning
Solution ·![4477.9780735698178_2D00_FB_5F00_732F5A13[1]](/assets/posts/2015/3/free-ebook-azure-machine-learning/4477-9780735698178_2d00_fb_5f00_732f5a131.png?w=245)
You’re into Machine Learning, got into Azure ML, looked at my couple of blogs about it and want to take it to the next level?Microsoft released an eBook for that exact purpose:
Free ebook: Azure Machine Learning (Microsoft Azure Essentials)
That book is targeted at...
-
Azure ML – Over fitting with Neural Networks
Solution ·In a past post, I discussed the concept of over fitting in Machine Learning. I also alluded to it in my post about Polynomial Regression.
Basically, over fitting occurs when your model performs well on training data and poorly on data it hasn’t seen.
In here I’ll give an example using Artificial Neural Networks. Those can be...
-
AzureML - Polynomial Regression with SQL Transformation
Solution ·I meant to illustrate over fitting (discussed in a past blog) with AzureML. An easy way to illustrate it is to fit a bunch of sample points near perfectly and the best tool for that is Polynomial Regression.
I was surprised to see that AzureML doesn’t support Polynomial Regression natively. But… while thinking about...
-
Azure ML - Simple Linear Regression
Solution ·Now that we got the basics of Machine Learning out of the way, let’s look at Azure Machine Learning (Azure ML)!
In this blog, I will assume you know how to setup your workbench.
In general, there are quite a few great resources for Azure ML:
-
Machine Learning - An Introduction - Part 2
Solution ·In a past blog entry I gave an overview of what Machine Learning is. I showed a simple linear regression example. The goal really was to explain to newcomer to the field what Machine Learning is, what type of problem it tries to solve and what the general approach is.
Of course, I used an extremely simple example in...
-
Machine Learning - An introduction - Part 1
Solution ·As I mentioned before, I did specialize (through graduated studies) in Machine Learning only to drop the field after a few years of trial on the Marketplace. I felt the field wasn’t ready for prime industrial applications.
Years have past, the field has matured and now is an exciting time to...
-
Twenty Years of Machine Learning at Microsoft
Solution ·Machine Learning is the new kid on the block.
This is of special interest to me since I specialized in that field 15 years ago to "unspecialized" three years later after being discouraged by the lack of real market of the discipline. Back then (early 2000's) ML applications were always academically inspired with little true business value delivered....
-
Skype Translator Infographic
Solution ·Microsoft just published an infographic about how Skype works its automatic speech recognition and translation (also known as Startrek universal translator).
I saw a demo of that technology 2 years ago, having an English and German speakers talking to each other but I didn't think it was that close to productization.
It...
![0[1]](/assets/posts/2015/3/strong-ai-existential-risks/01.jpg)
![4477.9780735698178_2D00_FB_5F00_732F5A13[1]](/assets/posts/2015/3/free-ebook-azure-machine-learning/4477-9780735698178_2d00_fb_5f00_732f5a131.png)
Mathematics
-
Quantum Computing - How does it scale?
Solution ·
We recently looked at Quantum Computing value proposition. We then looked at scenarios where Quantum Computing would have a big impact.Quantum Computer can solve some problems by several order of magnitude. This brings today’s intractable problems to be easy to solve tomorrow.
The key reason for...
-
Solution SLAs in Azure
Solution ·
Let’s talk about Service Level Agreement (SLA) of your solution in Azure.Hal Berenson wrote a great article about SLA lately. It is a great conceptual background for the present today.
Here we want to focus on how you should proceed to come up with an SLA for...
-
Hyperspheres & the curse of dimensionality
Solution ·I previously talked about the curse of dimensionality (more than 2 years ago) related to Machine Learning.Here I wanted to discuss it in more depth and dive into the mathematics of it.
High dimensions might sound like Physics’ string theory where our...
-
What is Statistics and why should you care?
Solution ·Unless you graduated in art, chances are you did a course in Statistics.Chances are you hated it.
Most people I know postponed that course until the end of their degree, didn’t understand much about it and hated it dearly.
I didn’t like it either and understood very little.
A few...
-
Where is the statistics in Machine Learning?
Solution ·I often try to explain what Machine Learning is to people outside the field. I’m not always good at it but I am getting better.
One of the confusion I often get when I start to elaborate the details is the presence of statistics in Machine Learning. For people outside the field, statistics are the stuff of survey or their...
Networking
-
Ingress rules in different Kubernetes namespaces
Solution ·In this article I want to show how an ingress controller in Kubernetes can be used to route traffic to workloads deployed in multiple namespaces.
The online doc for AKS deploys everything in the same namespace. Hence this article is a thin extension to the online doc.
The basic trick is to deploy...
-
Locking down Web App to Functions communications using subnets
Solution ·
Azure Functions are serverless Function as a Service.
Serverless compute typically offers less networking options since it does not run on dedicated compute.
Until recently, the only option to lock down access to a function was to lock down the function’s firewall to some public IPs. That isn’t always useful when the...
-
Azure App Service Authentication with Azure Front Door
Solution ·
Last time, we looked at Azure Front Door being used as a reverse proxy in front of Azure App Service.
We are going to increase the difficulty level today by looking at how to implement Azure AD authentication(also called easy auth) in Azure App Service behind Azure Front Door.
Although that...
-
Azure Front Door with App Service
Solution ·
Azure Front Door service was recently released.
Azure Front Door is an interesting service combining the capabilities of:
- Reverse Proxy (SSL Termination, URL based routing, URL rewrite & session affinity)
- Web Application Firewall (WAF)
- Accelerated Global routing
- Global Load Balancing between geo-distributed backend
- Some bits of Content Delivery Network (CDN, in the...
-
Multiple Service Endpoints to multiple services
Solution ·
Use case: I have a central thingy that needs to talk to a service protected by a service endpoints (e.g. storage account, Azure SQL DB, Azure Maria DB, etc.). That service is also accessible to another compute in other Virtual Networks.
Is that possible?
Just to make it a little more concrete, let’s...
-
Testing outbound connections in AKS-Kubenet and ACI
Solution ·What happens if a pod in AKS initiates a connection with a private endpoint? Which private IP address does the outbound connection uses?
This is relevant for a private IP inside the same VNET, a peered VNET or an IP accessible via a VPN or Express Route.
In general, the private IP of the...
-
Network Policies with Calico in AKS
Solution ·Calico network plugin is finally supported within Azure Kubernetes Services (AKS).
There is a very good tutorial on the online documentation, so we won’t give a walkthrough here.
Instead we will highlight a couple of points about Network Policies in general and in AKS.
Online references
On top of the aforementioned AKS...
-
Understanding multiple Ingress in AKS
Solution ·Last time, I covered the ins and outs of Ingress on AKS.
We looked at how to install nginx ingress controller and how the controller is deployed as a load balanced service. We did some URL based routing and domain name overloading.
This was in the spirit of clarifying the magic...
-
Understanding simple HTTP Ingress in AKS
Solution ·We looked at Kubernetes Ingress conceptually. We looked at different use cases: URL based routing and multiple domains.
We also looked at how ingress was implemented from an AKS perspective, i.e. where traffic was routed in nodes.
In this article, I wanted to get hands on. I figured we could start...
-
About Ingress in Azure Kubernetes Service (AKS)
Solution ·I did a bit of experimentation with Kubernetes Ingress, more specifically NGINX, lately.
I found the concept of Ingress utterly confusing at first. It is actually relatively simple. So, I thought I would share this sense of simplicity.
This article is conceptual. There will be no code nor even Portal tour.
If you want to ramp up on Ingress in...
-
AKS with Kubenet vs Azure Networking plug-in
Solution ·Update (22-03-2019): This configuration is now officially documented.
I’ve been diving into Kubernetes / AKS Networking lately. I thought I would share some of the insights I stumble upon.
We know AKS has two types of networking, basic & advanced, right?
- Basic provisions its own VNET and exposes only public IPs ...
-
Deploying AKS with ARM Template – Network integration
Solution ·In a past article, we looked at how Azure Kubernetes Services (AKS) integrated with Azure Networking.
AKS is a managed Kubernetes service in Azure.
In this article, we are going to do two things:
- Deploy an AKS cluster with Advanced Networking using an Azure ARM Template.
- Deploy a service on the...
-
Kubernetes Services in Azure AKS - Network integration
Solution ·Azure Kubernetes Services (AKS) is a managed Kubernetes service in Azure. Not only does it provides a managed cluster, it also integrates with Azure services.
In this article we’ll explain the integration between Kubernetes’ services and Azure Networking. We won’t give code sample though as we’ll keep that for a future article.
…
-
Using Application Gateway with VNET peering
Solution ·![pexels-photo-374710[1]](/assets/posts/2017/4/using-application-gateway-with-vnet-peering/pexels-photo-3747101_thumb.jpg "pexels-photo-374710[1]")
As I write these lines (early November 2017), Application Gateway doesn’t integrate well across VNET peering.More precisely, if we put the gateway in a VNET and have scale sets in another, the usual integration, where the scale set registers its VM automatically as the size of the set...
-
Azure Virtual Machines Anatomy
Solution ·
Virtual Machines can be pretty complex little beast. They can have multiple disks, multiple NICs in different subnets, can be exposed on the public internet either directly or through a load balancer, etc. .In this article, we’ll look at the anatomy of a Virtual Machine (VM): what are...
-
Virtual Network Service Endpoint - Hello World
Solution ·In our last post we discussed the new feature Virtual Network Service Endpoint.
In this post we’re going to show how to use that feature.
We’re going to use it on a storage account.
We won’t go through the micro steps of setting up each services but we’ll focus on the Service Endpoint configuration.
Resource Group
As usual for...
-
VNET Service Endpoints for Azure SQL & Storage
Solution ·
It’s finally here, it has arrived: Azure Virtual Network Service Endpoints.<p>This was a long requested “Enterprise feature”.</p><p>Let’s look at what this is and how to use it.</p><p>Please note that at the time of this writing (end-of-September 2017) this feature is available only in a few region in Public Preview:</p><ul><li>Azure...
-
Azure Application Gateway Anatomy
Solution ·Back in May, we talked about Azure Application Gateway.
In this article, we’re going to look at its anatomy, i.e. its internal component as exposed in the Azure Resource Manager (ARM) model.
A lot of Azure Resource has an internal structure. For instance, a Virtual Network has a collection of subnets.
Azure Application Gateway has a very rich internal...
-
URL Routing with Azure Application Gateway
Solution ·Update (13-06-2017): The POC of this article is available on GitHub here.
I have a scenario perfect for a Layer-7 Load Balancer / Reverse Proxy:- Multiple web server clusters to be routed under one URL hierarchy (one domain name)
- Redirect HTTP traffic to the same URL on HTTPS
- Have reverse proxy...
-
Joining an ARM Linux VM to AAD Domain Services
Solution ·Active Directory is one of the most popular domain controller / LDAP server around.
In Azure we have Azure Active Directory (AAD). Despite the name, AAD isn’t just a multi-tenant AD. It is built for the cloud.
Sometimes though, it is useful to have a traditional domain controller... in the cloud. Typically this is with legacy workloads built to work...
-
Troubleshooting NSGs using Diagnostic Logs
Solution ·
I’ve wrote about how to use Network Security Group (NSG) before.Chances are, once you get a complicated enough set of rules in a NSG, you’ll find yourself with NSGs that do not do what you think they should do.
Troubleshooting NSGs isn’t trivial.
I’ll try to give some guidance here but...
-
Virtual Machine with 2 NICs
Solution ·
In Azure Resource Manager (ARM), Network Interface Cards (NICs) are a first class resource. You can defined them without a Virtual Machine.UPDATE: As a reader kingly point out, NIC means Network Interface Controller, not Network Interface Card as I initially wrote. Don’t be fooled by the Azure logo ;)
Let’s...
-
Load Balancing VMs in Azure Resource Manager
Solution ·Here I want to show, in details, how you would go about to expose load balanced web server VMs using Azure Resource Manager (ARM) resources.
It sounds trivial but funnily enough I didn’t find an ARM template fully doing it without bugs.
I want to explain how it works and all the moving pieces (and there are a few). I’ll...
-
Using Network Security Groups (NSG) to secure network access to an environment
Solution ·Quite a few demos (including mines) ommit security for the sake of simplicity. One area where you can secure your applications in Azure is in terms of Networking.
Network Security Groups act as a firewall in the cloud.
In this post, I’ll show you how to create a virtual network with 3 subnets: front-end, middle & back-end. We’ll then secure...
![pexels-photo-374710[1]](/assets/posts/2017/4/using-application-gateway-with-vnet-peering/pexels-photo-3747101.jpg "pexels-photo-374710[1]")
NoSQL
-
Creating a Backup Solution for Cosmos DB using change feed
Solution ·Cosmos DB is Azure native NoSQL database. It has awesome capabilities such as global-distribution, exceptionally high availability, throughput scalability, and much, much more.
As with most NoSQL horizontally scalable databases it doesn’t have the same backup capabilities as mainstream RDBMS systems.
Cosmos DB has an automated backup...
-
Cosmos DB Stored Procedures - handling continuation
Solution ·I’ve recently did some work involving Stored Procedures in Cosmos DB.There are a few techniques to learn when our stored procedures handle large data sets. It is all about continuation.
In this article, we’ll look at how to build a robust and scalable stored procedure.
We’ll start with...
-
Cosmos DB Configuration Management
Solution ·I often found the need to automate Cosmos DB deployments.I like to have users run an ARM template and be ready to go. Automation is key component to repeatability. It enables automated testing and increase agility in many other ways.
ARM Templates don’t go beyond a database account.
…
-
Disaster Recovery with VM Scale Sets & Geo-Replicated DBs
Solution ·Last year we posted an article about different options available in Azure to implement a disaster recovery strategy.We strongly suggest to review that article as it gives good insights about what a disaster recovery strategy is within an already resilient Cloud Environment but also clear out a...
-
A few CosmosDB query limitations
Solution ·![pexels-photo-164524[1]](/assets/posts/2017/4/a-few-cosmosdb-query-limitations/pexels-photo-1645241_thumb.jpg "pexels-photo-164524[1]")
I’ve been working quite a bit with Cosmos DB since it was release in Private Preview (see my first article about it back in September 2014 as a battle scar).It has come a long way.
From DocumentDB, the NoSQL document-oriented DB to Cosmos DB, the Globally distributed...
-
Setup for populating Cosmos DB with random data using Logic Apps
Solution ·We recently published an article about Cosmos DB Performance with Geospatial Data.In this article, we’re going to explain how to setup the environment in order to run those performance test.
More importantly, we believe this article is interesting on its own as it shows how to use...
-
How to Create a Logic App Connector in an ARM Template
Solution ·I wanted to automate the performance test setup we discussed in the last article. This includes a Logic App calling into a Cosmos DB stored procedure.Logic App uses connectors when connecting to other services, either within or outside of Azure. Now how do we...
-
Cosmos DB Performance with Geospatial Data
Solution ·Time for some performance tests on Cosmos DB around Geospatial coordinates!Let’s hurt the service and see where it shines, where it has a hard time and how scaling it (i.e. increasing Request Units or RUs) helps.
We’ll publish an how-to create the same setup in a future article...
-
Invoking a Stored Procedure from a partitioned CosmosDB collection from Logic Apps
Solution ·I struggled a little to make that work, so I thought I would share the learning in order to accelerate your future endeavour.
I was looking at a way to populate a CosmosDB quickly with random data.
Stored Procedures came to mind since they would skip client-server latency. We can call a stored procedure creating hundreds of documents with random...
-
Hacking: changing Cosmos DB Portal experience from Graph to SQL
Solution ·In the last article, we looked at how we could access a graph using the SQL (aka DocumentDB) API.
Here we’ll explore how we can switch the Portal experience from one to the other.
Portal Experience
The Portal Experience refers to the way the portal lets us interact with Cosmos DB Data. It’s basically the Data Explorer experience.
Here...
-
Hacking: accessing a graph in Cosmos DB with SQL / DocumentDB API
Solution ·Azure Cosmos DB is Microsoft’s globally distributed multi-model database service.At this point in time (August 2017) there are four supported models: DocumentDB (also named SQL because the query language is similar to T-SQL), MongoDB, Tabular & Gremlin.
-
Cosmos DB Async Querying & Streaming
Solution ·I wrote an article back in January 2015 about async querying Azure DocumentDB using the .NET SDK.The service was still in preview back then.
Since then DocumentDB has been superseded by Azure Cosmos DB and the SDK has changed a bit so I thought I would...
-
Cosmos DB & Graph with Gremlin - Getting Started
Solution ·![gremlin-apache[1]](/assets/posts/2017/3/cosmos-db-graph-with-gremlin-getting-started/gremlin-apache1_thumb.png "gremlin-apache[1]")
Azure Cosmos DB is Microsoft’s globally distributed multi-model database service.One of the paradigm it supports is Graph: Cosmos DB can be used to store and query graphs.
At the time of this writing, it supports one interface, Gremlin, which is part of the Apache...
-
DocumentDB protocol support for MongoDB
Solution ·Microsoft announced, in the wake of many DocumentDB announcement, that DocumentDB would support MongoDB protocol.What does that mean?
It means you can now swap a DocumentDB for a MongoDB and the client (e.g. your web application) will work the same.
This is huge.
It...
-
Azure DocumentDB Demo
Solution ·December the 1st, 2015, I’m doing a presentation to a Montreal User Group, MS DEV MTL. Here is the script of each demo. Enjoy!
UPDATE: You can see the presentation slides here.
Account Creation & Adding Documents
For the creation of an Azure DocumentDB account, allow me to refer to myself in Creating an...
-
Major upgrade to Azure DocumentDB LINQ provider
Solution ·Early Septembre 2015, Microsoft has announced a major upgrade to the LINQ Provider of the .NET SDK of DocumentDb.I know it does appear a bit confusing since when DocumentDb was released (a year ago now), it was said that it supported SQL. Well, it supported some SQL.
Now...
-
Analysing Application Logs with DocumentDb
Solution ·Azure DocumentDB is Microsoft Document-centric NoSQL offering in the cloud.I’ve been working with it since Septembre 2014 and I wanted to share a use case I found it really good at: log analysis.
Now that takes some context.
I have been working for a customer using Microsoft Azure...
-
SQL Server 2016
Solution ·Here’s a rundown of my favourite new features in SQL Server 2016, largely inspired by the SQL Server Evolution video.
Impact of Cloud-First on SQL Design
This is a really nice illustration of the consequences of Cloud-First for Microsoft products.
SQL has been basically flipped around as a product. ...
-
Azure DocumentDB - Performance Tips
Solution ·![dt-improved-performance[1]](/assets/posts/2015/2/azure-documentdb-performance-tips/dt-improved-performance1.jpg?w=300)
Azure DocumentDB has been released for a little while now. Once you get passed the usual step of how to connect and do a few hello worlds, you will want to reach for more in-depth literature. Sooner or later, performance will be on your mind when you’ll want to take...
-
The limits of DocumentDB Preview Release
Solution ·
I was looking for the limits of DocumentDB Standard Tier, the only tier available during the preview release. It wasn’t all too trivial to find so here it is:http://azure.microsoft.com/en-us/documentation/articles/documentdb-limits/
Among the limits that may constrain your solution:
- Only 3 collections per Capacity Unit (but 100 Databases, the container of collections, per account) ...
-
DocumentDB Async Querying & Streaming
Solution ·UPDATE (31-08-2017): This article is superseded by the new article Cosmos DB Async Querying & Streaming.
Working with the .NET client SDK of Azure DocumentDB, I couldn’t find a way to query the store asynchronously.
***This post relates to the version 0.9.1-preview of Microsoft Azure DocumentDB Client Library. If you work with another major version, this might not...
-
Scaling a Multi-Tenant Application with Azure DocumentDB
Solution ·Following the release of Azure DocumentDB, Microsoft NoSQL document-oriented fully managed database, in preview mode, there was initial curiosity. Now we are starting to see more focused questions: how does it scale, can it expose OData, how would I go about architecture my department app around it as the back-end, etc. .
John Macintyre, Program Manager for Azure...
-
Querying Collections with DocumentDB Studio
Solution ·I released a first Release Candidate (RC) of DocumentDB Studio (release 1.0.0.1).
DocumentDB Studio is to Azure DocumentDB what SQL Management Studio is to SQL Server and SQL Azure: a one-stop shop to manage and interact with your DocumentDB.
I posted an installation guide of the application and an upgrade guide (both very simple).
…
-
DocumentDB Studio 1.0.0 released!
Solution ·Yes, the first non-beta version!
Ok, what is the difference between the betas and this one? Alpha, Beta and RC are always separated by arbitrary grass lanes…
For DocumentDB Studio, I was waiting to have a Minimal Viable Product (MVP) and for me the MVP was related to be able to query document collections.
Version 1.0.0...
-
Managing Documents with DocumentDB Studio
Solution ·I released a second Beta of DocumentDB Studio (release 0.2.0.1).
DocumentDB Studio is to Azure DocumentDB what SQL Management Studio is to SQL Server and SQL Azure: a one-stop shop to manage and interact with your DocumentDB.
I posted an installation guide of the application and an upgrade guide...
-
Upgrading DocumentDB Studio
Solution ·See this blog post on how to install Azure DocumentDB Studio from scratch.
Now with release 0.2.0.1 just out of the door, you might want to simply upgrade.
Upgrading is basically the same thing as installing, you just want to keep the file Studio.xml around.
That file keeps your account credentials and is compatible from release...
-
In Azure DocumentDB, DocumentClient.CreateDocumentQuery doesn’t exists!
Solution ·This is a quick help for you out there who are developing against Azure DocumentDB in .NET.
Maybe you read the Getting Started guide or you found that line of code somewhere else.
var families = client.CreateDocumentQuery(documentCollection.DocumentsLink, "SELECT * " + <span style="font-family:Times...
-
Managing Databases and Collections with DocumentDB Studio
Solution ·I released the first Beta version of DocumentDB Studio.
DocumentDB Studio is to Azure DocumentDB what SQL Management Studio is to SQL Server and SQL Azure: a one-stop shop to manage and interact with your DocumentDB.
I posted an installation guide of the application (very simple)....
-
Installing DocumentDB Studio
Solution ·As I announced yesterday, I released the first Beta version of DocumentDB Studio.
DocumentDB Studio is to Azure DocumentDB what SQL Management Studio is to SQL Server and SQL Azure: a one-stop shop to manage and interact with your DocumentDB.
In this blog post I'll simply walk you through the installation process.
…
-
Creating an Azure DocumentDB account
Solution ·In this post I will guide you on how to create an Azure DocumentDB account.
If you remember the Azure DocumentDB resource model, the entire resource model is rooted at the Database account:
So, step by step:
- Go to the preview portal: https://portal.azure.com/
- Click the NEW button at the bottom...
-
DocumentDB Studio first (beta) release
Solution ·I finally done a first release of my latest project, DocumentDB Studio!
DocumentDB Studio is to Azure DocumentDB what SQL Management Studio is to SQL Server and SQL Azure: a one-stop shop to manage and interact with your DocumentDB. The first release is quite minimalist and only let you:
- Connect to...
-
Profile of MSN Health and Fitness on Azure DocumentDB
Solution ·We already know that Azure DocumentDB was used in the back-end of OneNote. Last week Microsoft released another nice case study: the user profiles for MSN portal.
The requirements for the solution were:
- Scale requirements to support +425M unique MSN users with +100M direct authenticated users. Initial capacity requirements for 20TB of document storage.
- Under 15ms...
-
NoSQL implementation concepts
Solution ·While familiarizing ourselves with Microsoft new DocumentDB, a solid foundation on NoSQL doesn't hurt.
A few years ago, I saw a couple of great presentations on Channel9 (from a TechED if I am not mistaken) about Azure Storage. The presenters went into how partitioning works, how requests are routed, how consistency is ensure,...
-
Digest: DocumentDB Resource Model and Concepts
Solution ·Azure DocumentDB has release a few weeks ago and with it an early, in small quantity, of good quality documentation.
One of those article is DocumentDB Resource Model and Concepts. That article goes through the different concepts of the inner model of DocumentDB.
That article sheds some light on the product but also reveal the...
-
Azure DocumentDB: first use cases
Solution ·A few weeks ago Microsoft released (in preview mode) its new NoSQL Database: DocumentDB.
Not Only SQL (NoSQL) databases are typically segmented in the following categories: Key-Value (e.g. Azure Table Storage, Redis), Column (e.g. HBase, Cassandra), Document (e.g. CouchDB, MongoDB) & Graph. By its name but mostly by its feature set, DocumentDB falls in the document category.
…
![pexels-photo-164524[1]](/assets/posts/2017/4/a-few-cosmosdb-query-limitations/pexels-photo-1645241.jpg "pexels-photo-164524[1]")
![gremlin-apache[1]](/assets/posts/2017/3/cosmos-db-graph-with-gremlin-getting-started/gremlin-apache1.png "gremlin-apache[1]")
![dt-improved-performance[1]](/assets/posts/2015/2/azure-documentdb-performance-tips/dt-improved-performance1.jpg)
Office365
-
Unit Testing SharePoint
Solution ·Doing automated unit tests in SharePoint isn’t easy.
As with all libraries that haven’t been designed with unit testing in mind, SharePoint object model doesn’t expose its dependencies: it connects to a Content Database given the context creating it and there are no ways to redirect it to some stub implementations.
That is unless you can override method invocations. This...
-
Creating numbered headings or outline numbering
Gadget ·Far from fancy technological questions today…
How do you configure Microsoft Word to assign numbers to your section headers in a hierarchical way, e.g.:
- Header 1 (1)
- Header 1 (2)
- Header 2 (2.1)
- Header 2 (2.2)
- Header 3 (2.2.1)
etc.?
Well, maybe you’re in my position and you always had documents...
-
Office 365 Beta Service Descriptions
Solution ·
Microsoft has made available the descriptions of Office 365 Beta Services:http://www.microsoft.com/downloads/en/details.aspx?FamilyID=6c6ecc6c-64f5-490a-bca3-8835c9a4a2ea
I went through the Exchange Online Services & SharePoint Services. I noted the following features for Office 365 in general:
-
The road to Office 365
Solution ·Mary-Jo Foley has written a nice series of articles on her blog:
- The road to Microsoft Office 365: The past
- The road to Office 365: The present
- The road to Office 365: The future
In the series she traces back the origin of Office 365 (BPOS) into what it is today...
-
Windows 8: Desktop as a Service?
Solution ·In the wild country of rumours about Windows 8, there’s a new entry: Desktop as a Service (thanks to Mary-Jo Foley for the heads-up). Some slides have indeed leaked from the London Microsoft architectural summit in April 2010 showing Microsoft’s vision of the next step for Windows virtualization.
The virtualization of the...
-
Office Web App to power Facebook emails
News ·The new Facebook mail service will use Microsoft Office Web App in order to view Microsoft Office documents.
This follows the news of Facebook using Bing to search the social network.
It’s interesting to see Microsoft positioning itself in the social networking space and the search space. It recently abandoned the idea of powering Livre Blog by...
-
BPOS is dead, long live Office 365
Solution ·
Starting in early 2010, what was previously known as BPOS version 2 or Union will be available as Office 365. The name should reflect the availability of the platform (which suffered some reputation step back this summer). Today the beta is released, available to a 1000+ customers worldwide.Among...
-
BPOS SharePoint 2010
Solution ·
I finally read an official note from the BPOS team about a bunch of features I was looking for in BPOS, especially for SharePoint. Mainly:- Federated Identity!
- SharePoint 2010 Online
- Extranet access for SharePoint online
There are no date mentioned, but BPOS usually deploys new version quarterly.
As I mentioned in...
-
Microsoft Online (BPOS)
Solution ·I’ve been looking at the BPOS offering for my company. Since the documentation is quite spread-out on the web, I though a blog post about my findings would be useful.
Terminology
First thing, let’s clear out the terminology. As it is sometimes the case with Microsoft products, it’s not a simple thing.
Microsoft Online is MS offering for Cloud Services. ...
Operations
-
Tracing request in Azure API Management
Solution ·Azure API Management (API-M) is an API Gateway solution.
It is quite easy to create an API in API-M, connect it to a back-end API and test it in the portal.
What happen when you test it from another client platform (e.g. Postman) and it fails? You have no visibility into what...
-
Alerts on Azure Function failures
Solution ·
So, you have a few functions running. Maybe some of those functions are important and you would like to be alerted when they fail.
In this article I’ll cover that exact scenario. We will do that in the following steps:
- Deploy a recurrent function which fails %50 of the time (by design)
- Define...
-
Monitoring metrics in AKS
Solution ·AKS has a nice integration with Azure monitor. Out of the box there are a couple of dashboards for common metrics.
What if you need to go beyond those metrics?
This is what we’re going to do in this article. I’m going to show how to get the CPU usage per container. Along...
-
Creating a Backup Solution for Cosmos DB using change feed
Solution ·Cosmos DB is Azure native NoSQL database. It has awesome capabilities such as global-distribution, exceptionally high availability, throughput scalability, and much, much more.
As with most NoSQL horizontally scalable databases it doesn’t have the same backup capabilities as mainstream RDBMS systems.
Cosmos DB has an automated backup...
-
Custom Logs on AKS & Azure Monitor
Solution ·Let’s look at a concrete problem:
- I have containers deployed in AKS
- Those container log into custom files
- I want to analyse those logs using Azure Monitor (Log Analytics)
We’ll look at how to do that.
We leverage Azure Monitor for containers.
Custom logs in Log Analytics also is...
PowerShell
-
Finding a VM Image Reference, Publisher & SKU
Solution ·Automation is great.I love to script solutions in Azure.
But most of the time, that automation started with me fumbling and trialing different approaches in the portal. Once I got something around what I wanted I’ll automate.
I suspect you do the same thing.
This is why today we’re going...
-
Moving from Standard to Premium disks and back
Solution ·Azure Managed Disks (introduced in February 2017) simplified the way Virtual Machine disks are managed in Azure.
A little known advantage of that resource is that it exposes its storage type, i.e. Standard vs Premium, as a simple property that can easily be changed.
Why would we do that? Typically we’ll move from standard to premium storage to improve...
-
How to know where a Service is Available in Azure
Solution ·Azure has a Global Footprint of 40 regions at the time of this writing (mid-September 2017).Not all services are available in every regions. Most aren’t in fact. Only foundational services (e.g. storage) are available everywhere.
In order to know where a service is available, we...
-
Azure SQL Elastic Pool – Moving databases across pools using PowerShell
Solution ·I’ve written a bit about Azure SQL Elastic Pool lately: an overview, about ARM template and about database size.
One of the many great features of Azure SQL Elastic Pool is that like Azure SQL Database (standalone), we can change the eDTU capacity of the pool...
-
Finding ARM template ApiVersion
Solution ·
Writing an Azure ARM template for a Resource Group is getting easier every day but it remains a sport for the initiated.Here I want to give a tip about something I often find hard: how to get the API version of a resource in an ARM template?
As everything hardcore in the platform,...
-
Listing Resources under Resource Group with Azure PowerShell
Solution ·Simple task: I want to list resources under a single ResourceGroup within one of the subscription.
This is an excuse to show how to login on different subscription and play a bit with the PowerShell Azure SDK.
First thing, start PowerShell Integrated Script Environment (ISE) with the Azure SDK loaded.
Login
Login to your accounts…
<span... -
Move Azure Resources between Resource Groups using Powershell
Solution ·Ouf… I’ve been using Azure for quite a while in the old (current actually) portal. Now I look into my resources in the new (preview) portal and… what a mess of a resource group mosaic!
Unfortunately, at the time of this writing, you can’t move resources from a Resource Group to another via the portal…
If you’ve been there,...
Quantum
-
Quantum Computing - How does it scale?
Solution ·We recently looked at Quantum Computing value proposition. We then looked at scenarios where Quantum Computing would have a big impact.Quantum Computer can solve some problems by several order of magnitude. This brings today’s intractable problems to be easy to solve tomorrow.
The key reason for...
-
Quantum Computing - Scenarios
Solution ·We recently looked at Quantum Computing value proposition.Quantum Computer can solve some problems by several order of magnitude. This brings today’s intractable problems to be easy to solve tomorrow.
Not all problems have this property. In this article, we’ll look at some problems that do.
This is...
-
Quantum Computing - Value Proposition
Solution ·
Quantum Computing has gone from being an academic curiosity to a big bet in the last decade. Microsoft, Google, IBM and plenty of others are investing into that technology.
There still isn’t a large scale Quantum Computer commercially availably, but many think it’s around...
Security
-
Impersonating user in ADLS with Kusto
Solution ·We discussed Azure Data Explorer (ADX) and its query language Kusto in a past article.
In this article I wanted to show how to access an ADLS gen 2 using user impersonation, i.e. using the identity of the user running the queries to access the storage.
It would be all trivial and boring...
-
Locking down Web App to Functions communications using subnets
Solution ·Azure Functions are serverless Function as a Service.
Serverless compute typically offers less networking options since it does not run on dedicated compute.
Until recently, the only option to lock down access to a function was to lock down the function’s firewall to some public IPs. That isn’t always useful when the...
-
Accessing Azure Key Vault from within Azure API Management
Solution ·Today we look at a common although slightly advanced scenario with API Management: accessing Azure Key Vault from Azure API Management.
In an Enterprise, API Management service are often shared between teams. This means a lot of people might open it in the Portal and look at it. It also means that putting...
-
API Management - OAuth and private back-ends
Solution ·Azure API Management is a fully managed API Gateway service.
In my last article we looked at the anatomy of the service. Today I wanted to demonstrate how to use OAuth with JWT token to protect an API Front End. I also wanted to show how we can access backend APIs...
-
Azure App Service Authentication with Azure Front Door
Solution ·Last time, we looked at Azure Front Door being used as a reverse proxy in front of Azure App Service.
We are going to increase the difficulty level today by looking at how to implement Azure AD authentication(also called easy auth) in Azure App Service behind Azure Front Door.
Although that...
-
Azure Front Door with App Service
Solution ·Azure Front Door service was recently released.
Azure Front Door is an interesting service combining the capabilities of:
- Reverse Proxy (SSL Termination, URL based routing, URL rewrite & session affinity)
- Web Application Firewall (WAF)
- Accelerated Global routing
- Global Load Balancing between geo-distributed backend
- Some bits of Content Delivery Network (CDN, in the...
-
Authenticating an Azure service principal
Solution ·
When it comes to using Service Principal in Azure, I always advise using Managed System Identity (MSI).
MSI is simpler and safer. MSI handles certificate rotations. We never see the certificate. Remember this: the safest secret is the secret you never see.
But in cases we can’t use MSI, we are going...
-
Multiple Service Endpoints to multiple services
Solution ·Use case: I have a central thingy that needs to talk to a service protected by a service endpoints (e.g. storage account, Azure SQL DB, Azure Maria DB, etc.). That service is also accessible to another compute in other Virtual Networks.
Is that possible?
Just to make it a little more concrete, let’s...
-
Testing outbound connections in AKS-Kubenet and ACI
Solution ·What happens if a pod in AKS initiates a connection with a private endpoint? Which private IP address does the outbound connection uses?
This is relevant for a private IP inside the same VNET, a peered VNET or an IP accessible via a VPN or Express Route.
In general, the private IP of the...
-
Flex Volume in AKS
Solution ·I wanted to start looking at a few modules helping integrate AKS with the rest of Azure.
In a past article, we looked at Pod Identity.
This time around, we will look at Flex Volume.
Flex Volume is an integration between AKS and Key Vault. It allows keys, secrets & certificates (the...
-
Azure AD Pod Identity in AKS
Solution ·I wanted to start looking at a few modules helping integrate AKS with the rest of Azure.
A big integration point is identity. For many reasons, we’ll want our pods to use service principal identities:
- Access an Azure service supporting AAD-integration
-
Accessing Azure Key Vault using Managed Service Identity Logic Apps
Solution ·Azure Key Vault is a great service to manage secrets, keys & certificates.
It uses RBAC to control access. Like all access control system, there is a chain of access. For instance, my user account has access to the vault: this means if my account’s credentials get leaked, the access to the...
-
Service Principal for Logic App Connector
Solution ·Azure Logic Apps is a powerful integration platform.
It integrates with different services (inside and outside Azure) using connectors.
Connectors are responsible to authenticate to the service they represent.
Some connectors will hold the credentials. This is the case, for instance, of the SQL connector.
Other connectors will by default take the...
-
Creating a Service Principal with Azure CLI
Solution ·Service Principals are a bit of a weird beast. They are Azure Active Directory applicationswith kind of an extra bit. That bit says they can actually login by themselves. Hence the name principal. But being an application is kind of weird.
We covered Service Principals in the past. We covered how to...
-
RBAC and role assignment using ARM Templates
Solution · Azure supports Role Based Access Control (RBAC) as an access control paradigm.It allows to map a user (or a group of users) to a role within a given scope (resource, resource group, subscription or management group).
For instance, we could map my user identity to a Virtual...
-
Understanding identities in Azure AKS / Kubernetes
Solution ·We’ve recently looked at Azure AKS (Kubernetes Cluster Managed Services). We’ve looked at how to create a Kubernetes Cluster with 3 lines of Azure CLI.With this we are able to interact with the cluster & deploy containers.
There is a bit of “auto magic” happening in...
-
Security with API: OAuth, token-based access vs key-based access
Solution ·Let’s consider security with APIs, i.e how to securely identify the caller.There are two authentication methods quite popular in the cloud to secure APIs:
- Key-based access
- OAuth, or token-based access in general
Let’s compare them.
Key-Based
By key-based we mean...
-
Azure Functions HTTP – Authorization Levels
Solution ·In a past article, we looked at Serverless compute in Azure in general and Azure Functions specifically.In this article we wanted to focus on Azure Function triggered by HTTP requests and the different options we have to authenticate:
- Anonymous
- Function
- Admin
- System
- User
Those are...
-
Virtual Network Service Endpoint - Hello World
Solution ·In our last post we discussed the new feature Virtual Network Service Endpoint.
In this post we’re going to show how to use that feature.
We’re going to use it on a storage account.
We won’t go through the micro steps of setting up each services but we’ll focus on the Service Endpoint configuration.
Resource Group
As usual for...
-
VNET Service Endpoints for Azure SQL & Storage
Solution ·It’s finally here, it has arrived: Azure Virtual Network Service Endpoints.<p>This was a long requested “Enterprise feature”.</p><p>Let’s look at what this is and how to use it.</p><p>Please note that at the time of this writing (end-of-September 2017) this feature is available only in a few region in Public Preview:</p><ul><li>Azure...
-
Automating Role Assignment in Subscriptions & Resource Groups
Solution ·![keys-unlock[1]](/assets/posts/2017/2/automating-role-assignment-in-subscriptions-resource-groups/keys-unlock1_thumb3.jpg "keys-unlock[1]")
Azure supports a Role Based Access Control (RBAC) system. This system links identity (users & groups) to roles.RBAC is enforced at the REST API access level, which is the fundamental access in Azure: it can’t be bypassed.
In this article, we’ll look at how we can automate the role...
-
Managing Azure AD Application members in Portal
Solution ·One of Azure AD’s powerful concept is the application. It gives context to an authentication as we explained in this article.
An application can also be used as an authorization barrier since we can manage an application members. This is optional as by default, everyone in a tenant has access to its application. But if we opt in to...
-
Automating Azure AD
Solution ·In the previous article, we explored how to interact (read / write) to an Azure AD tenant using Microsoft Graph API.
In the article before that, we looked at how to authenticate a user without using Azure AD web flow.
Those were motivated by a...
-
Using Microsoft Graph API to interact with Azure AD
Solution ·In my last article, I showed how to authenticate on Azure AD using a user name / password without using the native web flow.
The underlying scenario was to migrate an application using an LDAP server by leveraging an Azure AD tenant.
The logical continuation of that scenario is...
-
Authenticating to Azure AD non-interactively
Solution ·I want to use Azure AD as a user directory but I do not want to use its native web authentication mechanism which requires users to go via an Active Directory page to login (which can be branded and customized to look like my own).I just...
-
Primer on Azure Monitor
Solution ·Azure Monitor is the latest evolution of a set of technologies allowing Azure resources monitoring.
I’ve written about going the extra mile to be able to analyze logs in the past.
The thing is that once our stuff is in production with tons of users hitting...
-
Securing REST API using Azure Active Directory
Solution ·Scenario: you have a web & mobile front-end, both using a REST API as a back-end. You want to secure that back-end with authentication / authorization. How do you do that in Azure?
There are obviously a bunch of ways to do that. In this post, I’ll discuss the recommended...
-
Multi-Tenant SaaS with Azure Active Directory B2B & B2C
Solution ·Scenario: I’m creating a Software as a Service (SaaS). I’m having multiple customers & I want to manage their identity. For some of my customers the users won’t have corporate identity ; I would like to offer them to login using their social identity (e.g. Facebook) if they want to or create an account on my site otherwise. For other...
-
Azure Active Directory Application
Solution ·This is a quick post to talk about what an Application is for Azure Active Directory (or Azure AD or AAD).
If you come from the on premise world and are used to Active Directory on Windows Server, the concept of an application within Active Directory probably is a bit foreign.
Think about how authentication works in the...
-
Row Level Security in 3-tiers applications
Solution ·Row-Level Security is a great new feature in SQL. It is already available in Azure SQL Database and will be available in SQL Server 2016.
This is Cloud-First for you.
Row-Level Security allows you to forbid a user to see and/or access data they shouldn’t be able to see. Traditionally this is implemented at the application level, for...
-
Azure Key Vault & SQL Server Connector Update
Solution ·Azure Key Vault is alive and well!The Azure service allowing you to store keys and secrets in a secured container has been released at the end of summer and it continues to improve.
The SQL Server Connector is a component that can be installed on SQL Server...
-
Azure Key Vault is now generally available
Solution ·Azure Key Vault is now generally available!
What does it change? Well, it gives us way more stability in the service in terms of API surface but also, it backs the entire service with an SLA!
You can read some of my past articles about Azure Key Vault:
-
Azure Key Vault - Pricing
Solution ·Azure Key Vault is an Azure packaged service allowing you to encrypt keys and small secrets (e.g. passwords, SAS) and manage them in a secure fashion. Azure Key Vault actually allows you to store cryptographic keys and do operations with them (e.g. encrypt data) without revealing the key, which is pretty cool. Check it out.
Now a common...
-
Azure Key Vault – Step by Step
Solution ·Azure Key Vault is an Azure packaged service allowing you to encrypt keys and small secrets (e.g. passwords, SAS) and manage them in a secure fashion. Azure Key Vault actually allows you to store cryptographic keys and do operations with them (e.g. encrypt data) without revealing the key, which is pretty cool. Check it out.
A typical problem...
-
Azure Key Vault
Solution ·Has somebody been peeking on my X-mas list?
Indeed, one of the weakness of the current Azure Paas solution I pointed out last year was that on non-trivial solutions you end up with plenty of secrets (e.g. user-name / password, SAS, account keys, etc.) stored insecurely in your web.config (or similar store).
I was suggesting, as a solution, to...
-
Surveying the Web API Security Landscape
Solution ·Ronnie Mitra published a nice review of the current security protocols & standards available to you to protect your API.
Ronnie gives a good overview of what used to be available, in SOAP WS-*, what is currently available (HTTPS, OAuth and the like) and what is brewing for the future (Oz & Macaroons).
-
How to improve Azure: Granularity of access
Solution ·In this blog series I explore some of the shortcomings of the Microsoft Azure platform (as of this date, April 2014) and discuss ways it could be improved. This isn’t a rant against the platform: I’ve been using and promoting the platform for more than four (4) years now and I’m very passionate about it. Here I am pointing...
-
How to improve Azure: Security Models
Solution ·In this blog series I explore some of the shortcomings of the Microsoft Azure platform (as of this date, April 2014) and discuss ways it could be improved. This isn’t a rant against the platform: I’ve been using and promoting the platform for more than four (4) years now and I’m very passionate about it. Here I am pointing...
-
How to improve Azure: Can you keep a secret?
Solution ·In this blog series I explore some of the shortcomings of the Windows Azure platform (as of this date, March 2014) and discuss ways it could be improved. This isn’t a rant against the platform: I’ve been using and promoting the platform for more than four (4) years now and I’m very passionate about it. Here...
-
Securing Azure Messaging Service Bus access
Solution ·I am currently working on a very exciting project involving systems integration across the Azure Messaging Service Bus. I thought I would share some of the painfully acquired knowledge nuggets with you.
About %90 of examples you'll find on Internet uses Azure Bus SDK with 'owner'. That is basically 'admin' privilege because owner...
-
New Microsoft Services Single Sign-In UX
Solution ·Microsoft is going to deploy a new simplified user experience to login in its different services.
You can opt-in to this new UX by going here. You will have to opt-in every week though.
-
Advertisement tracking in iOS 6 vs no-tracking in IE 10
Gadget ·As the Business Insider reported, Apple has quietly introduced a new technology called IDFA (Identification For Advisers) in iOS 6.
This technology is similar to cookies in browser and advertisers to know which iPhone user has looked at which ads on which web site. It is anonymous in the sense that no other information beyond your ID is...
-
Departmental Application Migration to Azure Blog Series
Solution ·Back in summer 2010 I did a proof of concept around Windows Azure & ADFS. The challenge was:
How can we deploy a departmental application in the cloud and have employees connecting on it using their corporate account?
Basically, how to project a corporate account in the cloud?
This was before Azure AppFabric Access Control. The solution had...
-
Everything you ever wanted to know about SQL Azure Connection Security
Solution ·See this TechNet article for a complete description of all the security topics:
http://social.technet.microsoft.com/wiki/contents/articles/sql-azure-connection-security.aspx
-
January 2011 Releases of the Identity Training Kit & the MSDN Online Identity Training Course
Solution ·You can read more details on Vittorio Bertocci’s excellent blog.
You can find the training kit here and the courses here.
The training kit is an update from last June’s and now contain elements about ADFS, Windows Azure AppFabric Access Control v 1.0 (Septembre 2010) and Access Control Lab (Decembre 2010).
The courses seem to be migrated...
-
Single Sign-On from Active Directory to a Windows Azure Application Whitepaper
Solution ·A new whitepaper on how to connect to Windows Azure hosted applications using Active Directory has been published in December 2010.
The two major building blocks are WIF & ADFS.
I did cover this topic in a series of blog last summer. Now you have a nicely packaged whitepaper explaining it!
Also, Microsoft has published a series...
-
ADFS 2.0 Step by Step and How To guides
Solution ·Microsoft released a series of document guiding on how to use ADFS 2.0:
http://technet.microsoft.com/en-us/library/dd727938(WS.10).aspx
However useful my blog series on how to setup ADFS 2.0 to be used with Windows Azure was, I’m sure those guides will also be useful ;)
-
Departmental Application Migration to Azure – Part 4 – ADFS with Azure web app
Solution ·This is part of a series of blogs. See the preceding blog entries:
- Departmental Application Migration to Azure – Part 1
- Departmental Application Migration to Azure – Part 2 – ADFS Installation
- Departmental Application Migration to Azure – Part 3 – ADFS with on-premise web app
As mentioned before on this...
-
ASP.NET + WIF article
Solution ·Here’s an MSDN article about how to integrate WIF into an ASP.NET application: Passive Authentication for ASP.NET with WIF.
It doesn’t talk about ADFS it goes into the details of the configuration of WIF. That’s very interesting since the STS Wizard generates a bunch of configuration but no explanation on what is doing what.
-
Expended feature set in Access Control Service (AppFabric LABS)
Solution ·Microsoft just released a new feature set on top of Access Control Service (ACS). It is currently in “incubation” in AppFabric Labs. The Azure Labs are charge-free developer previews of features. They should eventually move to “production” with charge patterns.
You can see a quick (5 minutes) demo of it on Wade Wegner blog’s. Wade is a Microsoft Technical...
-
Departmental Application Migration to Azure – Part 3 – ADFS with on-premise web app
Solution ·This is part of a series of blogs. See the preceding blog entries:
- Departmental Application Migration to Azure – Part 1
- Departmental Application Migration to Azure – Part 2 – ADFS Installation
For me, authentication is the major challenge for this proof of concept. The migration of our tiny database, the web application that...
![keys-unlock[1]](/assets/posts/2017/2/automating-role-assignment-in-subscriptions-resource-groups/keys-unlock13.jpg "keys-unlock[1]")
Serverless
-
Locking down Web App to Functions communications using subnets
Solution ·Azure Functions are serverless Function as a Service.
Serverless compute typically offers less networking options since it does not run on dedicated compute.
Until recently, the only option to lock down access to a function was to lock down the function’s firewall to some public IPs. That isn’t always useful when the...
-
Alerts on Azure Function failures
Solution ·So, you have a few functions running. Maybe some of those functions are important and you would like to be alerted when they fail.
In this article I’ll cover that exact scenario. We will do that in the following steps:
- Deploy a recurrent function which fails %50 of the time (by design)
- Define...
-
API Management exclusive access to Azure Function
Solution ·[Update 05-04-2019: Erratum on the original article. Logic Apps is actually able to perform public IP filering.]
Azure API Management acts as a front door to your APIs.
Typically, we do not want users / apps to be able to access the underlying APIs directly since that would bypass the API Management policies,...
-
Testing outbound connections in AKS-Kubenet and ACI
Solution ·What happens if a pod in AKS initiates a connection with a private endpoint? Which private IP address does the outbound connection uses?
This is relevant for a private IP inside the same VNET, a peered VNET or an IP accessible via a VPN or Express Route.
In general, the private IP of the...
-
My JSON gets serialized in a string - Logic Apps
Solution ·
Azure Logic Apps is a powerful tool.
It takes care of long running tasks, retries, bunch of integrations, etc . .
It also handles JSON payloads natively. JSON flows in and out of an app.
I recently faced a bit of a problem with JSON though.
My JSON was appearing serialized within a string....
-
Accessing Azure Key Vault using Managed Service Identity Logic Apps
Solution ·Azure Key Vault is a great service to manage secrets, keys & certificates.
It uses RBAC to control access. Like all access control system, there is a chain of access. For instance, my user account has access to the vault: this means if my account’s credentials get leaked, the access to the...
-
Using Azure DevOps REST API to start multiple releases with Azure Logic Apps
Solution ·In our last article, we looked at how we could leverage the Azure DevOps REST API to trigger multiple releases.
This is useful when rebuilding an environment: first we release the shared infrastructure then each service on top of it.
In this article, we’ll see an implementation example using Azure Logic...
-
Handling recursivity in Logic Apps
Solution ·Azure Logic Apps is a powerful integration platform.
Some workflows naturally land themselves to recursion. For instance, handling a hierarchical data structure often has a natural recursive solution.
Azure Logic Apps doesn’t allow recursion directly. It forbids a Logic App invoking itself.
In this article we are going to explore two avenues...
-
Service Principal for Logic App Connector
Solution ·Azure Logic Apps is a powerful integration platform.
It integrates with different services (inside and outside Azure) using connectors.
Connectors are responsible to authenticate to the service they represent.
Some connectors will hold the credentials. This is the case, for instance, of the SQL connector.
Other connectors will by default take the...
-
Implementing & Automating Azure Stream Analytics Pipeline
Solution ·In our last article, we set out to build a resilient architecture around streaming events.In this article, we are going to build the solution. We are going to use an ARM template which automates deployment. We will also go through the configuration of different services.
The goal...
-
Taming the fire hose: Azure Stream Analytics
Solution ·We came upon an interesting challenge recently.Let’s have a stream of events pouring in. The source is unimportant. Hundreds, peeking at thousands, of events per second.
We need to update a table in Azure SQL with a summary of the events.
Let’s just assume the events are about...
-
How to get rid of /api in Azure Function’s route?
Solution ·We looked at Azure Functions.We also looked at security around Azure Function used to implement APIs.
Something people will quickly notice when implementing an Webhook / API function is that its URL or route is always prepended by /api.
For instance, if we create a webhook...
-
Recurrent serverless batch job with Azure Batch
Solution ·
Most solutions have recurrent batch jobs, e.g. nightly / end-of-month batch jobs.There are many services we can leverage in Azure to run those. In this article, we are going to explore a service that has “Batch” in its name: Azure Batch.
Azure Batch is typically positioned...
-
Azure Functions HTTP – Authorization Levels
Solution ·In a past article, we looked at Serverless compute in Azure in general and Azure Functions specifically.In this article we wanted to focus on Azure Function triggered by HTTP requests and the different options we have to authenticate:
- Anonymous
- Function
- Admin
- System
- User
Those are...
-
Serverless compute with Azure Functions - Getting Started
Solution ·
I meant to write about Azure Functions & serverless computing for quite a while now. So here we go.This entry is meant as a “getting started” entry with pointers to documentation and general discussion. We’ll dive deeper in future articles.
Serverless
What is serverless computing?
…
-
Setup for populating Cosmos DB with random data using Logic Apps
Solution ·We recently published an article about Cosmos DB Performance with Geospatial Data.In this article, we’re going to explain how to setup the environment in order to run those performance test.
More importantly, we believe this article is interesting on its own as it shows how to use...
-
How to Create a Logic App Connector in an ARM Template
Solution ·I wanted to automate the performance test setup we discussed in the last article. This includes a Logic App calling into a Cosmos DB stored procedure.Logic App uses connectors when connecting to other services, either within or outside of Azure. Now how do we...
-
Invoking a Stored Procedure from a partitioned CosmosDB collection from Logic Apps
Solution ·I struggled a little to make that work, so I thought I would share the learning in order to accelerate your future endeavour.
I was looking at a way to populate a CosmosDB quickly with random data.
Stored Procedures came to mind since they would skip client-server latency. We can call a stored procedure creating hundreds of documents with random...
Streaming
-
Event Hubs ingestion performance and throughput
Solution ·Azure Event Hubs is a data streaming platform as a Service. It is an ingestion service.We’ve looked at Event Hubs as the ingestion end of Azure Stream Analytics in two recent articles (here & here).
Here we look at client side performance of different...
-
Implementing & Automating Azure Stream Analytics Pipeline
Solution ·In our last article, we set out to build a resilient architecture around streaming events.In this article, we are going to build the solution. We are going to use an ARM template which automates deployment. We will also go through the configuration of different services.
The goal...
-
Taming the fire hose: Azure Stream Analytics
Solution ·We came upon an interesting challenge recently.Let’s have a stream of events pouring in. The source is unimportant. Hundreds, peeking at thousands, of events per second.
We need to update a table in Azure SQL with a summary of the events.
Let’s just assume the events are about...
Team
-
Corporate Cultures
Leadership ·It is said that Netflix represents the new I.T. corporation well.
![Netflix-Xbox360-HiRes[1]](/assets/posts/2015/3/corporate-culture/netflix-xbox360-hires1.jpg?w=294)
If you are interested in seeing what their corporate culture looks like, have a look a the slide deck they show to their job candidates.
It has all the flair of the typical Silicon...
-
Team IQ
Leadership ·![iStock_team_magnifying_glass_000012495968XSmall[1]](/assets/posts/2015/3/team-iq/istock_team_magnifying_glass_000012495968xsmall1.jpg?w=300)
I’m still catching up on the articles I read months ago and wanted to share with you!
This one has been published in the New York Times and is about the intelligence of a team or the team IQ. It asks why some teams are...
-
50 Popular Business Books Summarized In One Sentence Each
News ·
Here is an excellent reading list of business books from Business Insider.Each book is summarised in one snappy sentence that makes it for quick reference.
The list contains both books about how to succeed as a business or an individual in the business world.
-
Large Projects
Solution ·There is something about large projects that you'll never find, hence never learn, in smaller projects. The complexity, both technical and in terms of people dynamics, creates an all new set of challenges.
I read the article I Survived an ERP Implementation – Top 10 Gems of Advice I Learned the Hard Way at the beginning of the...
-
Agile Trade-offs
Solution ·It was very refreshing to read Paul Dolman-Darrall’s article about the trade-offs to adopt in an agile delivery model.
Agile is more than 10 years old but still have whimsical attributes of a brand new artefact. I believe this isn’t unique to agile but tend to be the case for any delivery methodology. It seems that however how...
-
Solution Skill Quadrants–Part II, Solution Architecture
Solution ·In the last blog post I introduced my own patented Solution Skill model:
I won’t repeat the content of the last blog post, that’s what hyperlinks are for. The goal of this post is to address this simple question: as a Solution Architect, how can you help the situation?
…
-
Solution Skill Quadrants–Part I, the model
Solution ·For quite a while I’ve been having a model of skill requirements in my head. That model guided me to analyse situations in different companies, mostly the staffing patterns for the maintenance of applications. I wanted to formalize it a little more and hence this post.
I call this model the Solution Skill Quadrants. Now you might ask what this...
![Netflix-Xbox360-HiRes[1]](/assets/posts/2015/3/corporate-culture/netflix-xbox360-hires1.jpg)
![iStock_team_magnifying_glass_000012495968XSmall[1]](/assets/posts/2015/3/team-iq/istock_team_magnifying_glass_000012495968xsmall1.jpg)
Virtual Machines
-
Linux Custom Script - Docker Sandbox
Solution ·I do a lot of proof of concepts (POCs) as part of my job.I hate keeping demo environment around. They tend to become brittle, out-of-date and filled with the last stuff I did.
I prefer to start from a clean slate every single time when possible.
That means...
-
Finding a VM Image Reference, Publisher & SKU
Solution ·Automation is great.I love to script solutions in Azure.
But most of the time, that automation started with me fumbling and trialing different approaches in the portal. Once I got something around what I wanted I’ll automate.
I suspect you do the same thing.
This is why today we’re going...
-
Recurrent serverless batch job with Azure Batch
Solution ·Most solutions have recurrent batch jobs, e.g. nightly / end-of-month batch jobs.There are many services we can leverage in Azure to run those. In this article, we are going to explore a service that has “Batch” in its name: Azure Batch.
Azure Batch is typically positioned...
-
Disaster Recovery with VM Scale Sets & Geo-Replicated DBs
Solution ·Last year we posted an article about different options available in Azure to implement a disaster recovery strategy.We strongly suggest to review that article as it gives good insights about what a disaster recovery strategy is within an already resilient Cloud Environment but also clear out a...
-
Renaming Virtual Machine Disks
Solution ·![pexels-photo-208637[1]](/assets/posts/2017/4/renaming-virtual-machine-disks/pexels-photo-2086371_thumb.jpg "pexels-photo-208637[1]")
Let’s say we would like to rename disks on a Virtual Machine (VM). Here we mean renaming the Azure Resource Name of the managed disk. How would we go about that?Why would we want to? Primarily to get our internal nomenclature right. A typical example is when we...
-
Azure Virtual Machines Anatomy
Solution ·Virtual Machines can be pretty complex little beast. They can have multiple disks, multiple NICs in different subnets, can be exposed on the public internet either directly or through a load balancer, etc. .In this article, we’ll look at the anatomy of a Virtual Machine (VM): what are...
-
Virtual Network Service Endpoint - Hello World
Solution ·In our last post we discussed the new feature Virtual Network Service Endpoint.
In this post we’re going to show how to use that feature.
We’re going to use it on a storage account.
We won’t go through the micro steps of setting up each services but we’ll focus on the Service Endpoint configuration.
Resource Group
As usual for...
-
VNET Service Endpoints for Azure SQL & Storage
Solution ·It’s finally here, it has arrived: Azure Virtual Network Service Endpoints.<p>This was a long requested “Enterprise feature”.</p><p>Let’s look at what this is and how to use it.</p><p>Please note that at the time of this writing (end-of-September 2017) this feature is available only in a few region in Public Preview:</p><ul><li>Azure...
-
Moving from Standard to Premium disks and back
Solution ·Azure Managed Disks (introduced in February 2017) simplified the way Virtual Machine disks are managed in Azure.
A little known advantage of that resource is that it exposes its storage type, i.e. Standard vs Premium, as a simple property that can easily be changed.
Why would we do that? Typically we’ll move from standard to premium storage to improve...
-
Sizing & Pricing Virtual Machines in Azure
Solution ·
I’m recurrently asked by customers similar questions around sizing & pricing of Virtual Machines (VMs), storage, etc. . So I thought I would do a reusable asset in the form of this article.This is especially important if you are trying to size /...
-
Creating an image with 2 Managed Disks for VM Scale Set
Solution ·UPDATE (23-06-2017): Fabio Hara, a colleague of mine from Brazil, has published the ARM template on his GitHub. This makes it much easier to try the content of this article. Thank you Fabio!
We talked about Managed Disks, now let’s use them.
Let’s create an image from an OS + Data disk & create a Scale Set with...
-
Taking a snapshot of a Managed Disk
Solution ·We talked about Managed Disks, now let’s use them.
Let’s snapshot a Managed Disk and restore the snapshot on another VM.
Deploy ARM Template
{ "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "VM Admin User Name": { "defaultValue": "myadmin", "type": "string" }, "VM Admin Password": { "defaultValue": null, "type": "securestring" }, "VM Size": { "defaultValue": "Standard_DS4", "type": "string", "allowedValues":... -
Migrating from unmanaged to managed disks
Solution ·We talked about Managed Disks, now let’s use them.
Let’s migrate existing VMs from unmanaged to managed disks.
Install pre-requisites
Update your Microsoft Azure PowerShell to the latest bits in order to include Managed Disks functionalities
Deploy ARM Template
{ "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "VM Admin User Name": { "defaultValue": "myadmin", "type": "string" }, "VM Admin... -
Azure Managed Disk–Overview
Solution ·![pexels-photo-196520[1]](/assets/posts/2017/1/azure-managed-disk-overview/pexels-photo-1965201_thumb.jpg "pexels-photo-196520[1]")
Microsoft released Azure Managed disk 2 weeks ago. Let’s look at it!
What did we have until then? The virtual hard disk (.vhd file) was stored as a page blob in an Azure Storage account.
That worked quite fine and Azure Disks are a little more than that. A...
-
Joining an ARM Linux VM to AAD Domain Services
Solution ·Active Directory is one of the most popular domain controller / LDAP server around.
In Azure we have Azure Active Directory (AAD). Despite the name, AAD isn’t just a multi-tenant AD. It is built for the cloud.
Sometimes though, it is useful to have a traditional domain controller... in the cloud. Typically this is with legacy workloads built to work...
-
Troubleshooting NSGs using Diagnostic Logs
Solution ·I’ve wrote about how to use Network Security Group (NSG) before.Chances are, once you get a complicated enough set of rules in a NSG, you’ll find yourself with NSGs that do not do what you think they should do.
Troubleshooting NSGs isn’t trivial.
I’ll try to give some guidance here but...
-
Moving existing workloads to Azure
Solution ·
Applications born in the cloud can take full advantage of the cloud and the agility it brings.
But there are a lot of existing solutions out there that weren’t born in the cloud.
In this article I want to sketch a very high level approach on how...
-
Primer on Azure Monitor
Solution ·Azure Monitor is the latest evolution of a set of technologies allowing Azure resources monitoring.
I’ve written about going the extra mile to be able to analyze logs in the past.
The thing is that once our stuff is in production with tons of users hitting...
-
Single VM SLA
Solution ·
By now you’ve probably heard the news: Azure became the first Public Cloud to offer SLA on single VM.This was announced on Monday, November 21st.
In this article, I’ll quickly explore what that means.
Multi-VMs SLA
Before that announcement, in order to have SLA on connectivity...
-
Virtual Machine with 2 NICs
Solution ·In Azure Resource Manager (ARM), Network Interface Cards (NICs) are a first class resource. You can defined them without a Virtual Machine.UPDATE: As a reader kingly point out, NIC means Network Interface Controller, not Network Interface Card as I initially wrote. Don’t be fooled by the Azure logo ;)
Let’s...
-
Disaster Recovery with Azure Virtual Machines
Solution ·UPDATE 09-11-2017: See Disaster Recovery with VM Scale Sets & Geo-Replicated DBs for an example of the different concepts introduced here.
UPDATE 31-05-2017: This article was written in 07-2016. Today Microsoft announced Azure Site Recovery (ASR) support for Azure-to-Azure (in public preview). Options presented in this article are still valid but the assumption that ASR can’t help no is longer...
-
Recreating VMs in Azure
Solution ·
In this article I’m going to explain how to destroy VMs, keep their disks on the backburner and re-create them later.Why would you do that?
After all, you can shut down VMs and not be charged for it. You can later restart them and...
![pexels-photo-208637[1]](/assets/posts/2017/4/renaming-virtual-machine-disks/pexels-photo-2086371.jpg "pexels-photo-208637[1]")
![pexels-photo-196520[1]](/assets/posts/2017/1/azure-managed-disk-overview/pexels-photo-1965201.jpg "pexels-photo-196520[1]")
![<img style="background-image:none;float:right;padding-top:0;padding-left:0;display:inline;padding-right:0;border:0;" title="inclusion-229302_640[1]"...](/assets/posts/2015/3/soa-vs-mobile-apis/inclusion-229302_6401.png){kind=link}
Web
-
Ingress rules in different Kubernetes namespaces
Solution ·In this article I want to show how an ingress controller in Kubernetes can be used to route traffic to workloads deployed in multiple namespaces.
The online doc for AKS deploys everything in the same namespace. Hence this article is a thin extension to the online doc.
The basic trick is to deploy...
-
Azure App Service Authentication with Azure Front Door
Solution ·Last time, we looked at Azure Front Door being used as a reverse proxy in front of Azure App Service.
We are going to increase the difficulty level today by looking at how to implement Azure AD authentication(also called easy auth) in Azure App Service behind Azure Front Door.
Although that...
-
Azure Front Door with App Service
Solution ·Azure Front Door service was recently released.
Azure Front Door is an interesting service combining the capabilities of:
- Reverse Proxy (SSL Termination, URL based routing, URL rewrite & session affinity)
- Web Application Firewall (WAF)
- Accelerated Global routing
- Global Load Balancing between geo-distributed backend
- Some bits of Content Delivery Network (CDN, in the...
-
Azure Application Insights Role Name in web page
Solution ·We discussed using Role Name to separate telemetries from different applications in a previous article.
We discuss the Nuget package I developed to set the role name for each telemetry in a backend application.
In this article I wanted to cover another element that receives little coverage: how to do that for...
-
Azure Application Insights Role Name Nuget Package
Solution ·I’ve been using Azure Application Insights to monitor micro-services.
Since I have many micro-services, it isn’t practical to have an Application Insight resource per service. Given that those micro-services interact with each other and I want to monitor those interactions, it makes sense to get all telemetry in one place.
The issue with...
-
Understanding multiple Ingress in AKS
Solution ·Last time, I covered the ins and outs of Ingress on AKS.
We looked at how to install nginx ingress controller and how the controller is deployed as a load balanced service. We did some URL based routing and domain name overloading.
This was in the spirit of clarifying the magic...
-
Understanding simple HTTP Ingress in AKS
Solution ·We looked at Kubernetes Ingress conceptually. We looked at different use cases: URL based routing and multiple domains.
We also looked at how ingress was implemented from an AKS perspective, i.e. where traffic was routed in nodes.
In this article, I wanted to get hands on. I figured we could start...
-
Security with API: OAuth, token-based access vs key-based access
Solution ·Let’s consider security with APIs, i.e how to securely identify the caller.There are two authentication methods quite popular in the cloud to secure APIs:
- Key-based access
- OAuth, or token-based access in general
Let’s compare them.
Key-Based
By key-based we mean...
-
Azure Functions HTTP – Authorization Levels
Solution ·In a past article, we looked at Serverless compute in Azure in general and Azure Functions specifically.In this article we wanted to focus on Azure Function triggered by HTTP requests and the different options we have to authenticate:
- Anonymous
- Function
- Admin
- System
- User
Those are...
-
Using Application Gateway with VNET peering
Solution ·As I write these lines (early November 2017), Application Gateway doesn’t integrate well across VNET peering.More precisely, if we put the gateway in a VNET and have scale sets in another, the usual integration, where the scale set registers its VM automatically as the size of the set...
-
Azure Application Gateway Anatomy
Solution ·Back in May, we talked about Azure Application Gateway.
In this article, we’re going to look at its anatomy, i.e. its internal component as exposed in the Azure Resource Manager (ARM) model.
A lot of Azure Resource has an internal structure. For instance, a Virtual Network has a collection of subnets.
Azure Application Gateway has a very rich internal...
-
URL Routing with Azure Application Gateway
Solution ·Update (13-06-2017): The POC of this article is available on GitHub here.
I have a scenario perfect for a Layer-7 Load Balancer / Reverse Proxy:- Multiple web server clusters to be routed under one URL hierarchy (one domain name)
- Redirect HTTP traffic to the same URL on HTTPS
- Have reverse proxy...
-
Web Socket Protocol
Solution ·Web Socket is a new protocol, standardised in RFC 6455, attempting to bring the best features of HTTP & TCP together. More specifically, it aims at being Connected & full-duplex (as TCP), allowing server to call-back clients and universal (as HTTP).
This wasn’t done without pain. Web Socket has a non-trivial handshake process, done over HTTP, after which the...
-
Web Design: Make Complex thing Simple
Solution ·I’m always amazed at you a simple user interface can simplify complex tasks.
Take Windows Explorer where you can drag & drop multiple files from one folder to another. The user sees which folders the files are going into, it takes a few seconds and boom! Doing that by command line would be much more abstract, left alone much more...
-
Internet Explorer 9 – Beta Update
News ·
An update to Internet Explorer 9 Beta is available from Microsoft as of yesterday (November 23rd 2010).This is an update to the full browser as opposed to the developer preview build which isn’t the full Internet Explore, although the preview build does work side-by-side with any other version...
-
Microsoft’s jQuery contributions as official plug-ins
Solution ·A quick note about Microsoft’s contributions to jQuery (namely XYZ): they are now official plug-ins!
We’ve read several months ago about the contributions of Microsoft to jQuery, that is, Globalization, Templates and Data Linking.
Those plug-ins bring a programming model relatively close to Microsoft WPF (or Silverlight) but quite in line with jQuery. It also shows the...