Azure Databricks – RDD – Resilient Distributed Dataset

We looked at Azure Databricks a few weeks ago.  Azure Databricks is a managed Apache Spark Cluster service. In this article, we are going to look at & use a fundamental building block of Apache Spark:  Resilient Distributed Dataset or RDD.  We are going to use the Python SDK. It is important to note that … More Azure Databricks – RDD – Resilient Distributed Dataset

Recurrent serverless batch job with Azure Batch

Most solutions have recurrent batch jobs, e.g. nightly / end-of-month batch jobs. There are many services we can leverage in Azure to run those.  In this article, we are going to explore a service that has “Batch” in its name:  Azure Batch. Azure Batch is typically positioned for big compute since it easily schedule jobs … More Recurrent serverless batch job with Azure Batch

Azure Databricks – Getting Started

Apache Spark is rising in popularity as a Big Data platform.  It exists on this accelerated timeline for such an impactful technology. Think about it: 2009, started as a Berkeley’s University project. 2010, open sourced 2013, donated to Apache Foundation 2014, becomes Top-Level Apache Project In 2013, the creators of Spark founded Databricks.  Databricks has … More Azure Databricks – Getting Started

Security with API: OAuth, token-based access vs key-based access

Let’s consider security with APIs, i.e how to securely identify the caller. There are two authentication methods quite popular in the cloud to secure APIs: Key-based access OAuth, or token-based access in general Let’s compare them. Key-Based By key-based we mean an authentication scheme where we do pass a key to the API request. That … More Security with API: OAuth, token-based access vs key-based access

Azure Functions HTTP – Authorization Levels

In a past article, we looked at Serverless compute in Azure in general and Azure Functions specifically. In this article we wanted to focus on Azure Function triggered by HTTP requests and the different options we have to authenticate: Anonymous Function Admin System User Those are called Authorization Levels.  For each function in a function … More Azure Functions HTTP – Authorization Levels