Knowledge tells you it’s safe to push to prod in Fridays. Wisdom tells you not to. Experience raises your heart rate at the very thought (though, that might actually be ptsd)
My favorite personal fuck up was when I accidentally locked myself (and literally everyone else in the company) out of the CRM I was working on by disabling the login pages and enabling SSO before I had finished setting up the SSO inside the CRM’s config, and it logged me out as part of the procedure. Whoops.
Idk most teams I’ve worked with have either known better than to deploy anything at EOD or on a friday, or make heavy use of feature flags so any change that caused an issue just got swiftly rolled back. The ones that didn’t, I made it ABUNDANTLY clear that I won’t be available outside of work hours.
Maybe I haven’t been around the block enough or maybe I got lucky…
Probably depends on the type of company you work in. If it’s a long established one with lots of staff, they’ve probably realised this issue a long time ago and put plans in for it.
If it’s a more modern one that hired a bunch of solid old heads early on, they probably know better from the outset.
In both cases, someone, somewhere will have probably experienced it and said “never again”, so implemented (or improved) release procedures to ensure it doesn’t happen again
A lot of my teams have been on the younger side and for small companies/startups. So everyone either had a recent example to pull from or had first have experience
Remember, always, immediately, push new updates to prod, specifically right before you go home at the end of the day.
We’ve all done it. …right?
That’s how we know not to do it anymore
Knowledge tells you it’s safe to push to prod in Fridays. Wisdom tells you not to. Experience raises your heart rate at the very thought (though, that might actually be ptsd)
My favorite personal fuck up was when I accidentally locked myself (and literally everyone else in the company) out of the CRM I was working on by disabling the login pages and enabling SSO before I had finished setting up the SSO inside the CRM’s config, and it logged me out as part of the procedure. Whoops.
I think most people have done, or been part of a team that did, something similar.
At least most of the engineers I’ve worked with have had similar stories from their past
Idk most teams I’ve worked with have either known better than to deploy anything at EOD or on a friday, or make heavy use of feature flags so any change that caused an issue just got swiftly rolled back. The ones that didn’t, I made it ABUNDANTLY clear that I won’t be available outside of work hours.
Maybe I haven’t been around the block enough or maybe I got lucky…
Probably depends on the type of company you work in. If it’s a long established one with lots of staff, they’ve probably realised this issue a long time ago and put plans in for it.
If it’s a more modern one that hired a bunch of solid old heads early on, they probably know better from the outset.
In both cases, someone, somewhere will have probably experienced it and said “never again”, so implemented (or improved) release procedures to ensure it doesn’t happen again
A lot of my teams have been on the younger side and for small companies/startups. So everyone either had a recent example to pull from or had first have experience
Bonus points if you do it the day before you leave for vacation!