Continuous Delivery with Blue-Green Deployments


Blue-Green Deployment is a software pattern to deploy and release your application with minimal downtime and risk. This is achieved by maintaining multiple production-ready environments at the same time – termed as Blue and Green.

At any point of time, one of the environment is active & receives all the production traffic. When time comes for a new release, the changes are deployed to the other non-active identical environment. This can serve as the staging server and all the sanity testing is performed here. Once the changes are verified, you just flip the switch and all the traffic is now shifted to this new environment – which becomes the new Production environment. The previous Production environment still exists, and if there are any unexpected issues with the deployment, the traffic is shifted back to the old environment. So with just a flip of the switch we redirect Production traffic between 2 identical environments which is running the current and new version of the application.

Rolling back complex application and database changes is not easy. It might take hours of development team effort to attempt a clean rollback.

With Blue-Green Deployments, if there are issues with the deployment in Production, there is no need to spend hours planning and performing an application rollback. All we need to do is to modify the load balancer settings and shift all the traffic to the old server. There is no downtime, no effort lost in rollback and most importantly no impact on Business Users. It does not matter if it is during Business hours, because all you need to do is to shift traffic to your old server – which you know for sure works.

Blue-Green Deployment.png

Handling Database changes with this strategy might be a challenge, but it is something which can be mastered by following some best practices.
Keep in mind, sharing a single database is a better approach while doing a Blue-Green Deployment – instead of having 2 separate database and have strategies to keep the data in sync.

There are few things which you need to keep in mind while dealing with Database changes in a Blue-Green Deployment —

  • Don’t make destructive database changes – Do not drop a column. If you want to move data from 1 column to the other, do not delete data from the old column – which might break the old version of your application.
  • Ensure that your changes are backward compatible. If you want to add a new column, make the field nullable or add a default value so that both versions can run without issues.

I would highly recommend you to not make breaking changes and follow the ‘Expand and Contract Pattern‘ for your database changes to make sure that your database changes are backward compatible.

For example, if you want to make a breaking change like renaming a column name then follow the below steps —

  • Expand — Instead of renaming the existing column name, create a new column with the updated name.
  • Migrate — Move the data from the old column to the new column.
  • Contract — Once you verify that the code is functioning all right with the new column, then delete the old column name and make it non-existent.

Expand and Contract Pattern

Blue Green Deployments are an industry wide proven deployment strategy to increase reliability and uptime of your application. It is a change at a hardware level and not application level. However to prepare your application and make it ready for Blue-Green, you will need to adopt and make your application changes backward compatible. It is worth noting the fact that it comes with an additional cost of maintaining 2 Production ready servers/infrastructure, however the benefits easily overweigh the hardware cost.



Categories: DevOps, SQL Server

2 replies

  1. Is this click bait? I feel like I should invoice the author for my time and bandwidth. This is the line that just sums it all up. [Handling Database changes with this strategy might be a challenge, but it is something which can be mastered by following some best practices.] After explaining the magic behind blue-green (seriously, who comes up with these names? S…Oli>D, SCRUM, agile) he says that: “yeah, for databases that might not work”. Database updates ARE deployments. Nothing else is important. We already have all the tools we need. VM snapshots, web server configs, firewall/balancer changes. Hell, even a simple folder setup will work. It is the database updates that require synchronization and testing. Everything else is child’s play. Updating hundreds of databases and ensuring that schema drift is minimized is where the challenge lies, not in adjusting your DNS records to dump to server B. On top of the fluff, his advice on database changes is consultant advice.
    Make changes idempotent. Just think about that for a bit. Think about what it takes to decorate all your script with IF exists, and if obj_id() is not null scripts. Then you run the script and it blows because data problems. Are you just going to run it again? Hell, no. You have to find out what happened and why, back things you did out and so on. Now you have to test not only that your scripts are good, but that you can run them multiple times. That is insanity. Spend that time on being able to run the scripts on a restore again and again, then take a backup, then run the script. Or, I don’t know, use begin tran/rollback. It is built into the DB. You put those magic words before and after your script and presto, you can run it a million times. Don’t waste your time. Unless you bill hourly.
    Expand-migrate-contract pattern might be useful in some circumstance. Sure, if you are adding a middle name but only have a middle initial that might be a good strategy since old code and new code will look for two separate columns as you find out who is still using middle initial. However, if you are splitting address into address1 and address2 then you better believe you want your current production code to fail if you have not adjusted it. Otherwise you are just collecting junk data and are later going to be wondering, hmm how is this getting in the database?
    Devops is a euphemism for 90s “code and fix” spiced with general laziness.

    Like

    • wow. ultra sick burn pete! way to troll free advice. while i agree the database step is a little meh, the overall approach to blue green works well if you know what you are doing. if you take a dev approach to your operation, and arent too resistant to learning that, or niche’d into a role where you cannot scale, the old skool way is probably the only way. if this is your situation, sir, i feel bad for you.

      Like

Leave a Reply