Microsoft Ignite 2024 - My Key Highlights

Last week was the annual Microsoft Ignite conference in Chicago. In this blog, I will take you through my most important highlights of the sessions.

Fabric SQL Database (Preview): The future of SQL in the cloud

Let's get straight to the point with the biggest announcement: Fabric SQL Database! 

SQL Database in Microsoft Fabric is a developer-friendly, transactional database, based on Azure SQL Database, with which you can easily create an operational (OLTP) database in Fabric. In addition to the Fabric Lakehouse and the Warehouse, you now also have an efficient SaaS SQL database at your disposal as a developer with the performance of a transactional database and under the hood a(n extra) storage that is optimized for analytical purposes (Delta Parquet).

SQL Database in Fabric is:

• The home of OLTP workloads in Fabric
• Easy to configure and manage
• Set up for analytics by automatically replicating data to OneLake in near-real-time
• Based on the underlying technology of Mirroring in Fabric
• Under the hood, the same as Azure SQL Database, plus you have a web-based editor in the Fabric portal

With the introduction of Fabric SQL Database, Microsoft is taking traditional database management to a new level:

• Simple and secure architecture: Automated management and scaling optimizations
• Source control integration and CI/CD support: Essential for modern DevOps processes
• Capacity billing: Pausable compute resources ensure cost efficiency
• Unique cross-platform capabilities: Perform JOINs across databases, warehouses, and lakehouses

In addition to SQL Database in Fabric, the next version of SQL Server was also announced: SQL Server 2025. It is currently in Private Preview. It also supports mirroring to Fabric.

Metrics Hub: Standardized Metric management in Fabric (Preview)

The introduction of the Metrics Hub in Fabric is a new standard in how organizations define, track, and share KPIs. This new approach integrates seamlessly with Power BI and provides centralized management for metrics, enabling consistency and better collaboration. You might be thinking, this sounds familiar? You are! 😊 First there was Metrics, then Goals, and now the Metrics Hub where you can create Metric sets.

The advantage of being able to create and share metrics centrally is that you still have that one single version of the truth. Users can then use these measures in Power BI reports. But developers and data scientists can also use them in notebooks, using Semantic Link (Labs)!

Fabric AI Capacities

There will be an option to run all AI (=Copilot) compute on a separate capacity, so that other Fabric workloads are not affected by the use of Copilot. You still need at least an F64, and it's not available in all Azure regions, so be sure to check the Copilot FAQ for all your questions.

Fabric Org Apps

Org apps is a new item in your workspace, which now allows you to create an app as you did before by publishing an App in your workspace. One of the benefits of this is that you can create multiple Org apps per workspace!

In addition, you can customize more things like the design and behavior of the app. However, you can still only add items from the current workspace.

It is important to know that there is a tenant setting for Org apps that you need to activate. You can also use Org apps with a trial capacity.

Some other important updates for org apps compared to workspace apps:

• In addition to Power BI reports, you can now also add Fabric notebooks and real-time dashboards
• The distribution of org apps has been simplified, because access to underlying items (e.g. semantic models) is also applied, even in other workspaces. Users will therefore receive read access to all items that are dependent on the items in the app. These rights will also be removed when a user is deleted. There are a few exceptions, so check this.
• The items in an Org app are no longer a copy of the item, but the original items from the workspace
• Check all the differences between workspace and Org apps here

This week, an extensive blog about Org apps was also published where you can find even more information.

Power BI

• The biggest update by far: write-back with User Data Functions: native write-back integration with reusable functions and new options to link these to buttons in Power BI, with which you can update data in a source
• In the Pipeline activity, you can now refresh a semantic model at table and partition level
• Dataflow Gen2 updates: this item will soon also be supported in CI/CD (in source control and deployment pipelines)

OneLake

At the heart of Fabric is OneLake, a single central repository for all data, regardless of format. Some new developments:

• Shortcuts with Iceberg tables: Direct integrations with Snowflake and other platforms
• Mirroring: Support for Azure SQL and Snowflake (GA)
• Cross-cloud JOINs: Analyze datasets spread across different clouds and formats: You can combine SQL Database, Lakehouses and Warehouses in one query
• Open mirroring (preview): this allows you to write change data from any application directly into a mirrored database in Fabric. Open mirroring is based on the open Delta Lake table format

Platform

• Workspace monitoring (not to be confused with the Admin Monitoring workspace, this is intended for tenant insights): a database in your workspace that records log data and metrics about all items in that workspace. All of this is recorded in a real-time intelligence Eventhouse database. There are example queries in the Fabric samples GitHub. 
• Surge protection for capacities: This capability is coming soon, and it allows you to set a cap on the background workload of a capacity. This helps to better guarantee the availability of interactive workloads (such as Power BI report usage). You can set this per capacity.
• OneLake catalog: This is the new central hub to find, search, and use all Fabric items. You can use all available filters, including new ones like domains and tags. You can explore in-place items and view details.

Conclusion

Ignite 2024 showed that Microsoft is strongly committed to an integrated and future-oriented data platform, with AI and Copilot as a common thread. Innovations such as workspace monitoring, improved security through service principal support, and the advanced capabilities of the OneLake Catalog make it easier than ever to manage data effectively.

I believe that with these developments, the Microsoft Intelligent Data Platform is ready to support organizations in an increasingly data-intensive world.

Do you have any highlights from Ignite that I missed? Then let me know in the comments!


Items that are being worked on now and in the future are often public via the Fabric roadmap. There you can find per workload the items that will be worked on in the coming 6-12 months.

Some examples:

• A new Fabric navigation experience
• Support for folders for Git integration
• Better ownership of Fabric items
• Fabric Capacity Metrics Cross-capacity insights
• Capacity Metrics Chargeback

My DataGrillen Adventure: Speaking, Connecting, and New Friendships

I just got back from an incredible trip to DataGrillen, and I can’t wait to share my experiences with you. If you haven’t heard of it, DataGrillen is this amazing community-driven conference in Germany, where data professionals from around the world gather to share knowledge, ideas, and, of course, some good BBQ and beers. This year was special for me because I had the honor of speaking again at the event.

A special thanks goes out to Ben (@bweissman) & William (@sql_williamd) for organizing the event! But let's start from the beginning.

The Journey Begins

The anticipation was building as I packed up the car, later picked up Eduard Koekkoek (@heartforarts) and set off on our road trip to Germany. Eduard, a great friend and fellow data enthusiast, was the perfect companion for the journey. We spent some time on the road discussing everything from the latest in data technologies to our expectations for the conference. The drive was quite long, but the conversation made it fly by.

Stepping onto the Stage

The highlight of my trip was undoubtedly my speaking session. I had the first session at the first day, just after the opening "ceremony". The preparation that goes into a talk is always intense, but the passion I have for the subject kept me going. The topic was close to my heart, Administration and Governance in Fabric, and I was thrilled to see a room of fellow data enthusiasts ready to dive into the discussion. 

The audience was incredibly receptive, and the interaction was great. There’s something deeply fulfilling about sharing knowledge and seeing that spark of understanding and curiosity in others. The Q&A session that followed was lively, with thoughtful questions. It’s moments like these that remind me why I really like what I do.

Reconnecting with Old Friends

One of the best parts of conferences is the chance to reconnect with friends and colleagues from the industry. DataGrillen was no exception. With the change of forgetting someone, I won't call out names here 😀
It was like a reunion of sorts, seeing familiar faces, catching up on the latest in our professional and personal lives, and talking about past conferences and projects. There’s a special bond that forms when you share experiences and challenges over the years, and DataGrillen provided the perfect backdrop for these reunions.

A smaller group of people, specifically people into running, like to get together for an early run on days of a conference. We created a very special, long-running thread on Twitter (a.k.a. X), that actually started 2 years ago at DataGrillen!

Ran a little bit more 😊🥵 pic.twitter.com/dWM8mNwwo4

— Koen Verbeeck 🇺🇦 🌈 (@Ko_Ver) May 17, 2024

Furthermore, we exchanged ideas over beers and BBQ, debated the future of data technologies, and shared a lot of laughs. These moments are invaluable, and they remind me of the strength and warmth of the data community. It's not just about the work we do, but the friendships we forge along the way.

Making New Connections

While reconnecting with old friends was wonderful, making new ones was equally exhilarating. DataGrillen attracted a diverse crowd of professionals, each bringing unique perspectives and experiences. I met so many passionate individuals, each conversation was a learning opportunity, and a chance to see the data world through a different lens.

The DataGrillen Vibe

What sets DataGrillen apart is its atmosphere. The organizers have created an environment that fosters learning and networking in the most enjoyable way possible. The BBQ element adds a fun twist, making the conference feel more like a community gathering than a formal event. It’s the perfect balance of professional development and relaxation.

Looking Ahead

As Eduard and I drove back home, I felt a great sense of satisfaction and inspiration. DataGrillen was more than just a conference: it was a reminder of the incredible community we’re all part of. I left with new knowledge, strengthened friendships, and a bunch of new connections that I’m excited to nurture.

If you ever get the chance to attend DataGrillen, don’t hesitate. It’s an experience that goes beyond the sessions and keynotes – it’s about the people, the conversations, and the community. I’m already looking forward to next year’s event!

Session materials

My slides were already shared with the conference, but in case you haven't found them, you can find them at my GitHub.
I've also uploaded the template file I used in my demo for Admin Monitoring to the above page.

First page of my demo file

You can also find slides of previous conferences there. I generally follow this pattern: https://github.com/NickyvVr/talks/tree/master/slides/YYYY/yyyymmdd [Conference name]

Until next time, keep learning, keep sharing, and keep grilling!

Deprecation of Legacy Power BI Apps!

In case you missed the official blog post 2 months ago, I suggest you read my blog post :)

Or if you want you can refer to the official blog here: Announcing the retirement of legacy Power BI Apps (pre-audiences).

Already on March 6, 2023(!), Power BI apps with multiple audiences went Generally Available.

TL;DR

If you still have a pre-audience app, that is: an app that was created before app audiences where a thing, you can should upgrade your app now. Support from Microsoft will stop after April 30.

One other way you could also do the upgrade, is to unpublish your app and create the app again.

N.B. Be aware that:

• You will get a different URL (AppID) 
• Any custom changes that users did are gone, so e.g. personal bookmarks and comments

💡 Pro Tip: Only unpublish an app if you need to remove it! 

What does this actually mean?

Legacy apps will continue to work, also after May 1st. But technical support from Microsoft will stop after this date. So I do think it is a wise step to upgrade old apps.

Why do I want to upgrade?

• Content Management: In the new Power BI apps with audiences, you can have to add content via the Update app feature by navigating to the Content tab and clicking Add content. In the legacy app you used to include content by using the app toggle in the workspace. This toggle will not be supported anymore, although it is still shown in the workspace.
  
  

• Audiences: You can use different audiences for separate authorizations to a subsection of reports from the workspace. If you are using the default access request method, you also need to provide an audience group in the drop-down button with a pending access request before you can approve it.
• Content visibility: By default, all newly added content to an already published app is marked as invisible. App authors must manually unhide content for each audience group. 

More information on publishing apps, using audiences or current limitations can be found here: Publish an app in Power BI.

Upgrade process

In case you still have the legacy app, or if you haven't created an app in a workspace, the workspace might still show the old toggle button:

After you upgarde or create an (new) app, it will show either Yes, No or blank depending on the eligibility of the item to be in an app. That's because only Power BI items can be included in an app.

After you click upgrade, you'll get a notice the app was updated.

Be aware that after the upgrade your app is created withouth any audiences. So be sure to check the authorizations and shared content in the upgraded app.

Other updates to apps

Other recent updates to apps where announced here:

• The number of audiences has been increased (from 10) to 25
• You can now enable custom messaging for handling app access requests
• At the Fabric Community Conference in Las Vegas: Multiple apps per workspace coming soon!

Governance & Administration - Tenant Settings: Searching

With all the Fabric announcements in the last months, some of the Admin announcements might have slipped through. As you might know, the Admin part of Fabric is dear to my heart. I've posted about it earlier here, here, here, and here, to name a few 😀.

So in the next weeks I am going to highlight a few things with short, informative posts.

You can find other posts in the series here:

• Fabric (Tenant Settings) API's
• Tenant settings: Visual Cues

Governance & Administration - Tenant Settings: Visual Cues

With all the Fabric announcements in the last months, some of the Admin announcements might have slipped through. As you might know, the Admin part of Fabric is dear to my heart. I've posted about it earlier here, here, here, and here, to name a few 😀.

So in the next weeks I am going to highlight a few things with short, informative posts.

You can find other posts in the series here:

• Fabric (Tenant Settings) API's
• Tenant Settings: Searching
  

Governance & Administration - Fabric (Tenant Settings) API's

With all the Fabric announcements in the last months, some of the Admin announcements might have slipped through. As you might know, the Admin part of Fabric is dear to my heart. I've posted about it earlier here, here, here, and here, to name a few 😀.

So in the next weeks I am going to highlight a few things with short, informative posts.

This first post in the series is focused on the Tenant settings API and other Fabric API's available at the time of writing.

You can find other posts in the series here:

• Tenant Settings: Visual Cues
• Tenant Settings: Searching

Get Tenant Settings API

Obviously, to use the Get Tenant Settings API you need to have at least Tenant.Read.All permissions, or have the Fabric Administrator role (or higher) in Azure.

There are a few use cases I see for getting these settings exported with this API:

• Documentation purposes when you have multiple Fabric Administrators
• Distributing tenant settings to users, and explaining why we (as a team of Administrators/within the Center of Excellence) made certain choices
• Get notified of the changes in the tenant settings, without having to use Microsoft Defender or M365 Security & Compliance center like mentioned here

Get Capacities Tenant Settings Overrides

With the introduction of Fabric, there has also been the ability to have capacity admins override certain tenant settings. 

Example of a tenant setting that can be delegated

Some settings in the Admin portal are automatically delegated to capacity admins, like shown above. 

Other Fabric API's

Next to the API's on the tenant settings, there are already a lot more API's in the Admin section for you to check out, for example to handle workspaces, users, domains, and items as a Fabric Admin.

Next to that, so not as a Fabric Admin, you can also interact with Lakehouse tables, and there's a bunch of Core API's, for example to use Git or OneLake Shortcuts.

Preview note

Please note that at the time of writing this post, most of the API's are still in preview and subject to change.

Except for the Get Tenant Settings API, all the API's are currently in preview:

The Conference Season Has Begun!

The conference season is kicking off again! I have quite some events and sessions already lined up for the next months, so I though I'll update you here.

First up is Data Community Day Austria, in Vienna. I'm actually at Schiphol Airport now, waiting to board my plane 😀 I'm presenting a new session on Direct Lake in Microsoft Fabric.

In March, 7-9, the Dutch Power BI Gebruikersdagen are in Utrecht and I'm presenting a session on Saturday: How to use Power BI External Tools.

After that is SQLBits, which I'm skipping this year. MVP Summit is unfortunately also online for me this year, but that has a very good reason...

Because, in the last week of March the Microsoft Fabric Community Conference is happening in Las Vegas, and I'm delighted and excited to say I am presenting 2 sessions there!
The first session is with Ulrich Christ, Program Manager Azure Data Factory at Microsoft, and we are going to talk about Connecting to SAP Data with Data Factory in Microsoft Fabric.
Because the second session is still in the works I can't share any details yet, so you'll just have to wait a bit longer for that.

After that, there's also DataGrillen in Lingen, Germany on May 16 & 17. I'm presenting a session dear to my heart: Fabric Admin in a Day. Do check the amazing 2-day schedule on their site. It's also happening during the Kievelinge Fest, which happens the weekend right after DataGrillen, how convenient... 😁

I'm very excited for the coming months and hope to see many of you there, meeting up with old friends and making new ones! 💛