Software is quickly becoming integral part of human life as we see more and more automation and technical advancements. Just like we expect car to work all the time and can't afford to break or reboot unexpectedly, software industry needs to continue to learn better way to build software if it were to become integral part of human life. In this course, you will get an overview of how software teams work? What processes they use? What are some of the industry standard methodologies? What are pros and cons of each? You will learn enough to have meaningful conversation around software development processes. After completing this course, a learner will be able to 1) Apply core software engineering practices at conceptual level for a given problem. 2) Compare and contrast traditional, agile, and lean development methodologies at high level. These include Waterfall, Rational Unified Process, V model, Incremental, Spiral models and overview of agile mindset 3) Propose a methodology best suited for a given situation
Deployment: Cutover Strategies
Loading...
From the course by University of Minnesota
Software Development Processes and Methodologies
142 оценки
From the lesson
Software Development Processes : Part 2
In this module, we will learn about processes that are used to implement the software, verify and validate the software, deploy the software and maintain the software.
Meet the Instructors
Praveen MittalAdjunct Professor
College of Science and Engineering
Kevin WendtTeaching Specialist
Department of Computer Science and Engineering
[MUSIC]
Hi, in this video we're going to be talking about some deployment strategies,
in particular cutover strategies.
How we go about creating reliable system deployments, and especially updates.
So I'm going to talk a little bit about the potential cutover strategies and
tie them a little bit to my experience in the field.
So at the very bottom, the first cutover strategy
you should consider is cold back-up, sometimes also called cold storage.
Now this is the case where you really don't have anything ready, but
you have potentially, for example, the physical hardware.
You have a separate server ready to go.
So for example, if you happen to have geolocation separation
where you have a data center in, for example,
like we did here in Minnesota and another one in Arizona.
If the data center in Minnesota goes down,
you can make a call to the data center in Arizona.
They will spool up a physical machine, give you access.
You will go in and transfer all the data, transfer all the applications,
install the applications, configure.
And then potentially, depending on how well you have your data backed up,
do data replication onto this cold backup storage device.
Now from our perspective, when we were going through the idea of which cutover
strategy should we employee, we decided okay, how long is that going to take?
How much money is it going to cost us to keep a cold backup storage ready to go?
Which was relatively cheap.
But how long would it take us to recover from a failure?
If Minnesota's data center did go down and that took our entire application with us,
how long would it take to get the new version in, for example, Arizona, up and
running and working again in production-ready format?
We decided that it would take approximately 24 hours and
that seems to be a good average for something like a cold backup storage for
a large scale software system.
If you have thousands of transactions an hour,
that's a pretty big system that's going to be taking care of it.
It's likely going to take you 24 hours to get all the approvals,
all the installations, all the configurations up to date and then data
replication over, verify the installation, test it out, and then deploy it.
So we decided that that was going to take us about 24 hours.
That's what cold backup would look like.
We decided that was, in our instance of course since we're taking
thousands of transactions per hour, that was just not going to cut.
That's not going to be good enough for what we wanted to deploy in our system.
So we went one ring up the ladder.
Warm standby then is quite different.
Warm standby means that you actually have a running machine ready to go.
So you already have it apportioned.
You already have it, for example, installed on location or
in the example of virtual machines,
you have something where you could very easily one-click turn on the machine.
Now, this is where the definition gets a little bit fuzzy.
So it depends on what you mean when you say warm standby.
Some people when they say this mean that that's all it is.
It's the hardware that's there.
You still need to deploy and install.
There are some people that say it's actually there and when you turn it on,
all of the application installers, all the configuration is already in place.
You need to merely do the install or
do the configuration step to get it up and running.
So it depends who you talk to when you talk about warm standby.
Now again, that still means that our data wouldn't be there.
And since we're taking so many transactions in every time frame,
we still needed at least some data replication to move across.
Remember that this is warm standby.
It's not actually running, so it can't really do effective data replication.
Again, that's a bit of a grey area, but for our definition it wasn't going to be
accepting data replication, so we still needed to replicate all the data.
From a warm standby approach where we decided
it would mean that the machine was there and that, indeed, whenever we installed
a new version of production code, we would also update the warm standby.
We would turn it on, update the software, turn it back off.
So at least everything was installed and configured.
We could just turn it on and then somehow do data replication,
do any minor configuration changes that were necessary, and redirect the traffic.
We decided that in that instance, warm standby would take between two and
four hours.
We're including the amount of time it takes for us to decide that
there is an issue that's bad enough that we need to do warm standby.
Which could take up to 15 to 30 minutes depending on the person that's on call and
the time that it happens, to get the decision approved,
that yes we're moving to warm standby, as well as configuration, data replication,
turning it on, testing that it works before we launch truly production ready.
We decided it would be between two and four hours.
Again, we decided that was not good enough for our purposes.
When you're taking in thousands of transactions,
two to four hours is still thousands of transactions,
tens of thousands, potentially hundreds of thousands of transactions.
And that is a big risk.
We weren't ready to take on that risk.
So we went even one step further.
Hot failover is the final cutover strategy that we'll be talking about today.
A hot failover system is everything.
The machine or virtual machine is up ready and running.
The systems themselves are up, ready, waiting for
transactions but they just don't receive the information.
This is different than load balancing.
Load balancing says that you use a set of production servers that are all
supposedly identical and you distribute load across all of them.
A hot failover in its purest sense doesn't do that.
It means that it's truly a system that data does not go to even though it could
handle it at any moment.
This included for our purposes the definition that a hot failover for
us means that data was constantly being replicated.
So on a five minute delay, our data was constantly being replicated to our hot
failover site, even though transactions were not, I should say,
production live incoming requests were not going to the hot failover site.
The resulting data was being replicated across multiple sites.
So our hot failover site was sitting there, waiting for transactions.
That means that we could use our, I mean effectively a redirector, but you could
call it your load balancer, you could call it something else, merely to change
its configuration, to merely redirect the input to our hot failover site.
So, in a matter of minutes, we can make the call, decide and hit a switch,
potentially in a configuration file or a single database field, that automatically
redirects input from our external services into our new hot failover site,
so that it will take us, we imagine less than 30 minutes to make the switch.
We have a 15 minute call window for on-call production support.
For example, when it was my turn to be on call,
I had to be within 15 minutes of having my machine up and on and me logged in, so
no matter where I went, I couldn't be more than 15 minutes minus how long it takes
me to get my computer ready, away from my computer when I'm on call.
It takes another 15 minutes to decide and flip the switch.
We decided that that 30 minutes was good enough,
that the data that we would have to replicate would be small enough.
Only one or two 15 minute windows, which is how granular our time windows went,
would need to be additionally replicated if that data was lost in between
production going down and hot failover getting the redirect.
This is something that we use quite a bit.
And I'll talk about BCP, that's our business continuity plan.
None of these cutover strategies actually work unless you've tested it.
The worst thing you can do is have to go through one of these cutovers and
it not work.
So you have to test it.
In fact, for our purposes, we did a hot failover test once every quarter.
Every three months we actually hit a switch and
send everything over to hot failover, and it ran for a full day.
We usually also coordinated this with additional updates, and
that's how our production server got updated.
We would hot failover, make sure that worked, that was our BCP test.
Then we would update production,
you would hot failover back to production, make sure the updates work properly so
that we could roll back just by firing back over to hot failover.
If we decided that production was good, we would make the same
set of updates to hot failover, and again they would just be sitting there running.
Production taking data and hot failover just waiting for information.
There's a big difference between load balancing and cutover, and
especially when it comes to hot failover.
What we are really talking about is a geolocation separation.
That they're truly in separate places, and
that the hot failover does not receive data.
If you have a load balancing system that has geolocation separation, and
it's constantly sending data to multiple places,
we don't technically consider that a hot failover.
That's just geolocation separation in terms of load balancing, and
that's certainly good, and that very well could be a good way of doing it.
The problem from our perspective was, if for example one of them did go down,
are we sure and how do we know which data to replicate across?
That was one of the biggest problems that we had and we solved it by saying this
hot failover site is not being used, and all data can be replicated one direction.
That saved us a lot of bandwidth, a lot of processing, and
a lot of worry in how we set up our cutover process.
So I hope that these three cutover strategies are now a little bit more clear
and you have a good idea of how you might go about thinking about them if you wanted
to implement something like this for your system.
Ознакомьтесь с нашим каталогом
Присоединяйтесь бесплатно и получайте персонализированные рекомендации, обновления и предложения.
Coursera делает лучшее в мире образование доступным каждому, предлагая онлайн-курсы от ведущих университетов и организаций.
© Coursera Inc., 2018 Все права защищены.
