Avoiding ecommerce deployment disaster. Replatforming and deploying major updates are some of the most stressful moments for an ecommerce team. Replatforming and deploying major updates are some of the most stressful moments for an ecommerce team. These moments are vital for staying ahead of the competition, for introducing innovative new features or responding to user testing, but they’re also the point at which things can go most wrong.
Too often when you or your agency throw the hypothetical switch you end up with a site that’s got serious bugs or, even worse, no site at all.
What can you do to ensure that the deployment of your new platform, or of important revisions to your existing one, run seamlessly and effectively?
Always have a roll back strategy
A roll back strategy is the way to prepare for the unexpected, giving you a second chance to undo the deployment and go back to where you started.
Having a clear strategy to undo your changes is vital, however, before you put it into action, remember that sometimes fixing the problem can be quicker than rolling it back.
Accepting an amount of downtime whilst you fix an issue rather than instigating a time consuming rollback and going through the whole process again can sometimes be the better option.
Also, some deployments, especially of major changes, may include changes to the database. This is something that isn’t included in a version control system rollback.
Check payment processes
One of the most common area issues can emerge in a newly deployed site is in payment processes. Payment integrations can be difficult to properly test in the build process so make sure you do manual tests again and again before you go live.
There is also software available to test payments in a mimicked production environment and if you’re introducing new payment processes, then this sort of technology is vital.
Have at least four development environments
Having separate desktop (developer’s computer), development server, staging server and production server environments gives you the freedom to develop whilst maintaining control of final code and allows you to refine deployment scripts as code is pushed through the environments.
It means you can test at each stage without disrupting the development cycle.
It’s also important to make sure that all of your environments, including developers’ local setups, are running the same versions of software and hardware so you can be sure that the code is operating consistently at each step of the process.
This avoids the all too common ‘but it works on my machine!’ excuses.
Think about deployment timing
If it’s a big deployment that involves planned downtime (or the risk of unplanned downtime), do it out of hours if possible.
If it’s an international site operating in multiple time zones then there may not be a ‘down’ time, in which case choose the least worst timeslot based on historical trading patterns.
It might seem counterintuitive to take a site down in the middle of the day UK time but, if a lot of your business is coming from the Far East, this might make the most sense.
Never deploy on a Friday
It can be tempting to get an update out before the weekend but, quite apart from the fact that Friday can be a busy trading day, it can also set you up for hard-to-solve issues over the weekend when people can be difficult to reach.
Problems can sometimes take hours or even days to emerge, potentially meaning that your site could go down out of hours like early on a Sunday morning.
Deploying earlier in the week means that issues are less likely to happen at inconvenient times.
Consider database processes
If you have setup scripts in your deployment, be aware of database processes that could take a long time to undertake.
For example, if you’re adding a column to a database with a million rows it may take less than a second for each row to update, but that time adds up.
Test deployments with production levels of data before you deploy so you have a realistic idea of how long the deployment is going to take, enabling you to plan realistically for potential downtime.
Communicate
When it comes to any deployment there should be an open line of communication between you, your agency, your management, your marketing and your developers.
Retailers should know about the potential impact of deployments, how long they could take and what might go wrong.
Also, agree fixed deployment slots in advance to stop people pushing for inappropriate, rushed deployments. This also minimises the risk that the marketing department could send out a 50% off voucher at the same time as you’re deploying your changes.
Minimise the size of deployments
It’s not always possible, but doing less, more often is usually the best approach.
Breaking up one big release into lots of smaller deployments means you can more quickly identify where issues are coming from if they emerge. It will also break up downtime into smaller pieces, lessening the overall impact on consumer experience.
Document
Documenting is something everyone should do, but it’s easy to forget or ignore.
By using a ticketing and bug tracking system to document your code, you can check commits against ticket numbers in order to ensure that all issues are resolved before deployment.
This approach not only enables a rigorous checking process, it also feeds into your rollback strategy making sure you know exactly what each commit was for should key team members leave/go ill/fall under a bus.
Have all the code that needs to be deployed ready
It sounds obvious but, without the right development processes and policies in place, it’s all too easy for a developer to forget to send all of the code to the server prior to deployment.
Adherence to proper development protocol, along with rigorous checks and testing along the way, will means that complete code should be ready to switch from staging to production each and every time.
You can never completely de-risk a deployment. However, by taking some simple logistical steps you can prepare for the worst whilst also doing your best to ensure the worst never happens!
