Microsoft Fabric launched with a promise that was easy to explain and much harder to deliver. Bringing ingestion, lakehouse, warehouse, BI, real-time analytics and data science into one SaaS platform sounded compelling. Early releases could still feel like a large catalogue where each workload moved at its own pace.
In September 2026, the number of tiles in the portal is no longer the interesting part. Operational coherence is. Can a team collaborate, version, test, deploy, configure and monitor without inventing a different process for every item? Fabric has made a significant leap on that front.
Fabric is becoming a much more credible enterprise data platform. CI/CD now covers the main building blocks, new Dataflow Gen2 items are versionable by default and Runtime 2.0 brings Spark 4.1. Caution is still required because several important features remain in preview and item dependencies do not all rebind automatically.
The most important change is not another tile
The major improvement is a shared foundation. Fabric REST APIs sit at the bottom. Git connects workspaces to Azure DevOps or GitHub. Deployment pipelines promote content across Dev, Test and Prod. Variable libraries carry stage-specific configuration. The Fabric CLI, Terraform and the fabric-cicd library complete the automation layer.
None of these elements is revolutionary on its own. Together they change the product. Fabric is starting to behave like a platform that can be operated with normal DataOps practices rather than a collection of experiences that teams move manually.
| Change | Status on September 1, 2026 | Practical impact | Watch point |
|---|---|---|---|
| Fabric Runtime 2.0 | GA, still opt-in | Spark 4.1, Delta 4.2, Python 3.13 and Java 21 | Default rollout is planned for late September |
| Dataflow Gen2 with Git and CI/CD | Default for new items | Integrated versioning and deployment | Connections still need a dedicated strategy |
| Branched workspaces and selective branching | Preview | Feature-level development isolation | Dependencies and excluded items require discipline |
| Mapping Data Flow in Dataflow Gen2 | Preview | Visual transformations running on Spark | Connector and transform coverage is still limited |
| Warehouse source control | Preview | Versioned SQL objects as a DacFx project | Validate before using it in a critical release path |
| OneLake storage tiers | GA | Hot, cool and cold lifecycle management | Policies must remain aligned with BI and Spark access patterns |
CI/CD finally becomes a coherent platform
Microsoft now documents a complete CI/CD architecture. Git, deployment pipelines and variable libraries build on the same APIs. Automation can run in GitHub Actions, Azure Pipelines or an internal runner. Service principal support prevents a release process from depending on a developer's personal identity.
The new Git experiences go further. A branched workspace isolates a feature branch in a workspace linked to its source. Selective branching avoids copying an entire domain when a developer only needs a few items. Rich code comparison also makes changes easier to review before a commit or synchronization. These features remain in preview, but they address a real problem in large workspaces.
The bulk publish mode in fabric-cicd 1.2.0 uses the batch import API where supported and falls back to standard publishing elsewhere. It is a quiet but meaningful improvement for platforms deploying dozens of items. CI/CD for SQL analytics endpoints and Warehouse source control are also moving forward with DacFx projects and incremental schema deployments.
Item definitions are versioned, not data. Some dependencies rebind through logical IDs, while others require a post-deployment action. Connections, secrets, permissions and parameters need an explicit configuration plan. They are not an automatic consequence of committing the workspace.
I still prefer a simple delivery model. The development workspace is connected to Git. A pull request carries review and tests. Promotion to Test and Prod uses dedicated workspaces, non-human identities and explicit environment values. The Fabric interface can make daily work faster, but the repository remains the source of truth.
Dataflow Gen2 changes category
Dataflow Gen2 used to expose one of the largest gaps between convenient authoring and industrial operation. Since April 2026, every new item is created with Git and CI/CD support. Existing non-CI/CD Dataflow Gen2 items keep running, but they are no longer offered for new creation. Conversion uses Save As.
The authoring cycle is simpler as well. Save replaces the old split between saving and publishing. Just-in-time publishing keeps what the editor shows aligned with what the next refresh will execute. Deployment pipelines then move the dataflow across workspaces.
Configuration remains the real challenge. Public parameters can change logic, sources or destinations at runtime. Variable libraries centralize stage-specific IDs and values. Relative references reduce hard-coded GUIDs. Microsoft still documents that source connections cannot be dynamically reconfigured like ordinary variables. Teams therefore need to validate each stage and prepare the required bindings.
Another major development is Mapping Data Flow Transforms. It brings visual, Spark-backed transformations into Dataflow Gen2. Teams can author low-code transformations at greater scale and progressively migrate Mapping Data Flows from Azure Data Factory or Synapse. The feature remains in preview, but it finally brings the Power Query world closer to a distributed engine inside one experience.
- Git and deployment pipelines. New dataflows join the same release lifecycle as other Fabric items.
- Incremental refresh. Data is split into buckets and only changed buckets are replaced in supported destinations.
- Fast Copy. Large ingestion workloads can use a parallel copy path instead of the classic mashup engine.
- Modern Evaluator. The .NET 8 evaluator is generally available and can accelerate selected scenarios.
- Refresh diagnostics. Detailed diagnostic package downloads are in preview for faster failure analysis.
Dataflow Gen2 is no longer merely Power Query hosted in Fabric. It is becoming a front end that can combine the mashup engine, SQL staging, Fast Copy and Spark compute. That flexibility is valuable, but it makes CU cost analysis more important. Two visually similar dataflows can follow very different execution paths.
Spark 4.1 and Runtime 2.0 change the foundation
Fabric Runtime 2.0 became generally available in August 2026. It includes Apache Spark 4.1, Delta Lake 4.2, Python 3.13, Java 21, Scala 2.13 and Azure Linux 3.0. It is still opt-in as I write. Microsoft plans to make it the default for new workspaces and environments in late September.
I would not switch an entire production workspace without an intermediate stage. A dedicated Environment lets teams test Runtime 2.0 on selected notebooks and Spark Job Definitions. Python libraries and wheels need to be republished in some environments after the Python upgrade. Delta Lake 4.x-specific features are still experimental and should not be enabled on tables that need to remain interoperable across every Fabric workload.
The Native Execution Engine is likely the most visible performance improvement. It offloads supported operators to a vectorized C++ path built on Apache Gluten and Velox. Microsoft reports up to six times faster performance and about 83% compute savings against open-source Spark in a representative TPC-DS benchmark. That result is useful evidence, not a universal promise.
The native engine is particularly effective with Parquet and Delta on expensive aggregations, joins and transformations. It now accelerates CSV, Python and Scala UDFs, and complex types. Structured Streaming, JSON, XML and ANSI mode still do not use that path. Unsupported operators fall back to the JVM. The meaningful indicator is therefore not whether the toggle is enabled, but what the Spark UI shows in the executed plan.
| Component | Runtime 1.3 | Runtime 2.0 | Consequence |
|---|---|---|---|
| Apache Spark | 3.5.5 | 4.1 | New SQL, PySpark and streaming capabilities |
| Delta Lake | 3.2 | 4.2 | Liquid Clustering and faster snapshots, with interoperability caveats |
| Python | 3.11 | 3.13 | Library validation is mandatory |
| Java / Scala | 11 / 2.12 | 21 / 2.13 | A substantial JVM toolchain refresh |
| Native engine | Available | Available and expanded | Acceleration without rewrites for compatible operators |
Default and Early Access release channels are another useful improvement. They let teams validate library, security and operating-system changes before those updates become standard. Custom Live Pools, still in preview, reduce startup latency for workloads needing prewarmed Spark capacity. Capacity-level concurrency and queue monitoring is now GA, which helps connect perceived slowness to actual contention.
Pipelines and operations gain visibility
Data Factory is progressing through the removal of small operational frictions rather than one dramatic feature. Notebook activities can populate their base parameters automatically. Activity-level monitoring lets operators drill from a pipeline run into the failed activity. Refresh SQL analytics endpoint is GA, while materialized lake view refresh can be orchestrated in a pipeline in preview.
Microsoft is also pushing migration from Synapse and Azure Data Factory. AI-assisted skills can assess assets, migrate notebooks, Spark Job Definitions and pipelines, then convert workspace global parameters into variable libraries. I would use that capability as an inventory and transformation accelerator, never as an architecture sign-off. A faithful migration can reproduce decisions that should have been reconsidered.
Operations Agent for Pipelines, currently in preview, monitors failures and abnormal durations and sends notifications to Teams. The concept is relevant. Before rolling it out broadly, I would verify signal quality, agent permissions, exposed data and the operational cost of false positives.
OneLake and Warehouse mature without as much noise
OneLake now provides item-level size reporting and lifecycle management across hot, cool and cold tiers. This is a practical FinOps improvement. A unified platform does not remove the need to know what is stored, for how long and for which workload.
Zero-copy distribution patterns are becoming stronger through shortcuts and OneLake security. Delegated shortcuts, still in preview, can use an organizational account, service principal or workspace identity to reach the target. This creates useful cross-domain and cross-tenant scenarios, but permission intersections need to be understood before becoming a standard pattern.
Materialized lake views now expose run analytics, failure trends and recommendations. Event-driven refresh remains in preview. Lakehouse Query Explorer is GA and makes it easier to test Spark SQL before moving logic into a notebook or SQL endpoint.
Warehouse IDENTITY columns are now GA. Source control and GPU query acceleration remain in preview. GPU acceleration looks impressive on paper, with Microsoft reporting up to seven times faster eligible queries under high concurrency. I still want to see workload eligibility, cost and stability before turning that headline into a general rule.
What I would do now on a Fabric platform
- Map items and dependencies. Separate what Git, deployment pipelines and APIs support from what still needs manual action.
- Standardize Dev, Test and Prod. Give each stage its own workspace, identity, connections and documented variable set.
- Use the CI/CD model for new Dataflow Gen2 items. Convert existing dataflows progressively with Save As, not as a Friday-night bulk migration.
- Test Runtime 2.0 in a dedicated Environment. Replay critical notebooks, wheels, Delta writes and reads from other workloads.
- Measure the native engine. Enable acceleration on a representative batch, then compare duration, CU and fallback rates in the Spark UI.
- Treat previews as previews. A valuable feature can enter a pilot without instantly becoming a production standard.
- Track cost by execution path. Dataflow, Spark, Warehouse and Fast Copy use different meters. A simple portal must not hide consumption.
This approach may look conservative, but it avoids two extremes. The first ignores capabilities that are now mature enough to matter. The second turns every new portal feature into an enterprise standard before anyone reads its limitations.
My verdict
Fabric is finally becoming more interesting to operate than to present. That is my best summary of this release wave. The product still has a huge surface and a difficult number of previews to track. Yet the components that determine day-to-day reliability are moving in the right direction.
CI/CD no longer stops at a handful of Power BI items. Dataflow Gen2 joins a normal release lifecycle. Spark adopts a modern runtime and a credible native engine. OneLake improves storage, security and observability. Warehouse is moving closer to the SQL engineering practices teams expect from a proper development chain.
I would not claim that every issue is solved. Connections, dependencies, uneven maturity and Delta interoperability still require architectural work. In 2026, however, Fabric is no longer only the promise of a unified platform. It is starting to provide the mechanisms required to govern and deliver one properly.
Quick questions
Is Microsoft Fabric CI/CD now complete?
The foundation is far more coherent with Git, deployment pipelines, variable libraries, APIs, the Fabric CLI and infrastructure-as-code tooling. Coverage and limitations still vary by item type.
Should teams migrate to Fabric Runtime 2.0 immediately?
Runtime 2.0 is generally available but remains opt-in until its planned default rollout in late September 2026. Teams should validate libraries, environments and Delta interoperability on representative workloads first.
Are new Dataflow Gen2 items versioned in Git?
Yes. Since April 2026, all new Dataflow Gen2 items are created with Git and CI/CD support. Existing non-CI/CD dataflows keep working and can be converted with Save As.
Does the native engine accelerate every Spark workload?
No. It accelerates many operators and formats, but unsupported work falls back to the JVM engine. Structured Streaming, JSON, XML and ANSI mode remain among the documented limitations.
Sources
- Microsoft Learn, what's new in Microsoft Fabric.
- Microsoft Learn, Fabric CI/CD overview.
- Microsoft Learn, CI/CD workflow options.
- Microsoft Learn, branched workspaces and selective branching.
- Microsoft Learn, Fabric variable libraries.
- Microsoft Learn, Dataflow Gen2 with Git and CI/CD.
- Microsoft Learn, ALM architectures for Dataflow Gen2.
- Microsoft Learn, Mapping Data Flow Transforms in Dataflow Gen2.
- Microsoft Learn, incremental refresh in Dataflow Gen2.
- Microsoft Learn, Fabric Runtime 2.0.
- Microsoft Learn, Native Execution Engine for Spark.
