EF Core migrations mostly just work. But in a team environment you might have come across cases when the migration you just created contain operations from earlier migrations. The case for this is almost always that the model snapshot was corrupted before. The most common reason for this is that remove migration is actually not working that well and can often remove too much from the snapshot. I'll show how this happens and how you can fix / avoid it.
You can find the code for this article at https://github.com/MikaelEliasson/EFCoreSnapshotCorruptionDemo
Some background and short theory
With EF6, using migrations in a team environment was quite painful because you had to handle parallel migrations on different feature branches by creating "merge migrations". Which was basically adding a new migration that was completely empty.
EF Core tried to solve this by adding the model snapshot which is added to the migrations folder and is a snapshot of your DbContext at the time you ran the last migration. On this small demo project it looks something like this.
The beauty of this is that because it's a single file that you will check into source any parallel migrations are merged together. And in case you have conflicting migrations you will actually get a merge conflict.
How does this work? Well, when we have the snapshot above, which is for a model with only this entity.
public class Blogpost
{
public Guid Id { get; set; }
public string Title { get; set; }
public DateTime PublishDate { get; set; }
public string Text { get; set; }
public string Authors { get; set; }
}
And we then add another property and add a new migration, EF Core will compare your new DbContext to the ModelSnapshot. Once it figures out that there is a property in the DbContext that doesn't exist in the ModelSnapshot it generates a migration that adds this column to the database. But it also updates the snapshot to match the context that this migration was based on.
Quite straigforward in most cases. That is when we do everything correctly and only go forward. What about when we make an error an have to go back? EF Core have the remove migration command for that. At it's core it does two things.
- It deletes the migration files from your project.
- It reverts the snapshot to the previous version.
To know what the previous version should be, each migration is stored with a copy of the model snapshot. I'll call this copy for migration snapshot in the rest of the post. See the *.designer.cs file that is stored with any migration for an example.
This is the last migration and you can see that it matches ContextModelSnapshot.cs perfectly when it comes to the model it builds.
So if I had added a new migration that I wan't to remove on top of this state and then run remove migration. EF Core would copy the snapshot of the migration in the image above to ContextModelSnapshot.cs and then delete the newer migration, leaving 20210116072020_AddAuthors.cs as the last migration again.
Ok, that was how it's supposed to work. Let's dive into where it breaks.
So what causes EF Core to generate faulty migrations?
The TL;DR version of that is that model snapshot is incorrect so when EF Core does the diff it sees more changes than what should be there. And the model snapshot most likely got corrupted when you did a remove migration and the previous migration had a migration snapshot that was missing parts of the snapshot, meaning it copied a faulty snapshot to your model snapshot.
But how? Let's look at the change graph of the project where I simulated that two persons work on the code in parallel.
In the example project the following things happened
Initial model (version:Parent)
|
|
Two persons start to work in parallel
-----------------------------------
Branch A | Branch B
| | |
Version A | Version B
(add text prop) | (Add publish date prop)
| | |
| | |
---- MERGE ---->| |
| |
|<--- MERGE --------
Merged version
|
Version C
Add new migration (faulty)
|
Version D
remove migration from C
|
Version E
Add new migration again (correct)
I'll show the interesting parts. But here are links to the specific commits for anyone that want to dive deeper.
So let's take what happens step by step
1. Parent version
This version is correct. We have this entity
public class Blogpost
{
public Guid Id { get; set; }
public string Title { get; set; }
}
and this snapshot (20210116070542_Initial.Designer.cs is identical so I don't post that)
modelBuilder.Entity("EFCoreSnapshotCorruptionDemo.Blogpost", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("uniqueidentifier");
b.Property<string>("Title")
.HasColumnType("nvarchar(max)");
b.HasKey("Id");
b.ToTable("Blogposts");
});
2. Version A
This version is correct. We have this entity
public class Blogpost
{
public Guid Id { get; set; }
public string Title { get; set; }
public string Text { get; set; }
}
and this snapshot (20210116070832_AddText.Designer.cs is identical so I don't post that)
modelBuilder.Entity("EFCoreSnapshotCorruptionDemo.Blogpost", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("uniqueidentifier");
b.Property<string>("Text")
.HasColumnType("nvarchar(max)");
b.Property<string>("Title")
.HasColumnType("nvarchar(max)");
b.HasKey("Id");
b.ToTable("Blogposts");
});
3. Version B
This version is correct when it's commited. But will be incorrect once it's merged.
We have this entity
public class Blogpost
{
public Guid Id { get; set; }
public string Title { get; set; }
public DateTime PublishDate { get; set; }
}
and this snapshot (20210116071004_AddPublishDate.Designer.cs is identical so I don't post that)
modelBuilder.Entity("EFCoreSnapshotCorruptionDemo.Blogpost", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("uniqueidentifier");
b.Property<DateTime>("PublishDate")
.HasColumnType("datetime2");
b.Property<string>("Title")
.HasColumnType("nvarchar(max)");
b.HasKey("Id");
b.ToTable("Blogposts");
});
4. Merged version
This version is incorrect. It correctly merges the model snapshot so it contains for Text from A and PublishDate from B.
See that merge commit;
However, notice how the migration snapshot for 20210116071004_AddPublishDate.Designer.cs have no clue to what happened in branch A.
We are completely missing the Text property in that snapshot. And because the migrations was done in an order so 20210116071004_AddPublishDate is the last one. That migration snapshot is what will be used if you add a new migration and then do a revert.
5. Version C
Let's add a new migration on top of the old ones. (We will later remove this). But this version is actually correct and fixes the issue caused by B for later migrations.
public class Blogpost
{
public Guid Id { get; set; }
public string Title { get; set; }
public DateTime PublishDate { get; set; }
public string Author { get; set; }
}
and this model snapshot. The migration snapshot (20210116071819_AddAuthor.Designer.cs) is actually identical here. Remember how it copies to old model snapshot into the migration. That's why it doesn't matter here that the previous migration snapshots was missing info.
modelBuilder.Entity("EFCoreSnapshotCorruptionDemo.Blogpost", b =>
{
b.Property<Guid>("Id")
.ValueGeneratedOnAdd()
.HasColumnType("uniqueidentifier");
b.Property<string>("Author")
.HasColumnType("nvarchar(max)");
b.Property<DateTime>("PublishDate")
.HasColumnType("datetime2");
b.Property<string>("Text")
.HasColumnType("nvarchar(max)");
b.Property<string>("Title")
.HasColumnType("nvarchar(max)");
b.HasKey("Id");
b.ToTable("Blogposts");
});
6. Version D
Now we figured out that we want multiple authors so we want to change Author -> Authors. Let's start with reverting the migration in Version C. This will make the Model snapshot incorrect because it miss information.
As you can see it actually removes the Text property from the model snapshot. This is even though we have this entity.
public class Blogpost
{
public Guid Id { get; set; }
public string Title { get; set; }
public DateTime PublishDate { get; set; }
public string Text { get; set; }
public string Author { get; set; }
}
Note: The author property should have been removed too but I forgot. But that doesn't matter for the example.
So now we are in a state where our DbContext has the Text property but the snapshot hasn't. Let's see what happens when we add the fixed migration.
7. Version E
This has an incorrect Up() method in the migration. It tries to add the Text column which was already added by 20210116070832_AddText.cs.
In 20210116072020_AddAuthors we have this Up() method
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<string>(
name: "Authors",
table: "Blogposts",
type: "nvarchar(max)",
nullable: true);
migrationBuilder.AddColumn<string>(
name: "Text",
table: "Blogposts",
type: "nvarchar(max)",
nullable: true);
}
But the Text column operation has already run before with 20210116070832_AddText.cs. This will create SQL errors when trying to apply the migrations.
From 20210116070832_AddText.cs
protected override void Up(MigrationBuilder migrationBuilder)
{
migrationBuilder.AddColumn<string>(
name: "Text",
table: "Blogposts",
type: "nvarchar(max)",
nullable: true);
}
That's it really, all the steps to get into this faulty step. Which is something we run into at Bokio regularly.
How to fix it?
I have no idea on how to fix the root cause of this but I have learned how to prevent and fix it when it happens. Some suggestions:
Do a separate commit for each migration.
remove migration doesn't really do anything git can't do for you. It's just restoring files to a previous state and git already have this.
And when you didn't commit?
If you are like me and get carried away and forget to commit as often as you should you will likely run remove migration followed by a new add migration and realise that this happened when you review the changes and be in a state where you have a bunch of changes packed together.
What I do then is that I run remove migration right away to get rid of the new faulty migration. Then I use git to either just rever the model snapshot to the correct model or manually copy pasting it from the history if I messed up too much to just revert it.
After that you can add your migration again it should be fine. It sounds harder than it is. If you understand how the snapshot works it fairly OK to reason about how you should fix this.
Bonus: Preventing the corrupted snapshot from being commited to keep your teammates happy.
The way we do this at Bokio is to have a unit test that checks that the model snapshot is up to date with the DbContext. Specifically we check this by asking EF Core if there is any migrations pending to be created. If there is none we are fine.
I'll try to blog this and add a link to our implementation.
Summary
We have seen that EF Core has trouble with remove migration in some quite common situations. Try to use git instead of the remove migration command. And if you happen to run into this issue you should be able to solve it fairly easy using the steps above.