• Access Control in Azure Data Lake Storage

  Solution  · 16 Jul 2020

  

  About a year ago I did an article about Azure Data Lake Storage (ADLS) gen 2 and how to use its REST API.

  ADLS gen 2 unlocked a bunch of scenarios. It is present in every region, it enables analytics operations (e.g. renaming folders), it supports HDFS protocol, it scales, etc. .

  In...

• Kusto Ingestion REST API

  Solution  · 02 Jul 2020

  

  Yes, this week we have Kusto & a Kitty. Can’t get better than that ;)

  We discussed ingestion in Azure Data Explorer / Kusto at length in past articles. We mentioned queued ingestion along the ride without diving much into it. Let’s do that now.

  Queued ingestion is available in the SDK...

• Azure Databricks Overview workshop

  Solution  · 24 Jun 2020

  

  I haven’t written about Databricks for quite a while (since April 2018 actually) but I had the pleasure to lead an Azure Databricks workshop with a local customer recently.

  For that I prepared quite a few demos now available on GitHub.

  I covered quite a few angles so I thought it would...

• My ultimate ingestion process for historical data at scale with Kusto

  Solution  · 10 Jun 2020

  

  In my last article, we discussed different architecture aspects of large historical data ingestion.

  In this article, I want to be more prescriptive and share an approach that works well for me.

  Is that the ultimate process? Of course not, that is clickbait. As we discussed at length in the previous article, different...

• Ingesting historical data at scale with Kusto

  Solution  · 03 Jun 2020

  

  There are many ways to ingest data in Kusto.

  There is batching vs streaming, queued vs command, plugins, SDK, etc. . There is also a plethora of tools / techniques to achieve this, e.g. Azure Data Factory, LightIngest, .ingest into, etc. .

  In this article I want to look...