Yeah every IT dept is sending out that notification. Someone replied with 'When do you expect it to be back online?" to which I LOL emoted at the moment. I think I'm a jerk. But how can you ask such a question- as if they planned a time to restore services and aren't working to restore it as quickly as possible?
Powercycling is a designed method to bailout Cisco routers. If you botch a configuration command and lose connection, you can have someone powercycle it to restore you last good configuration. Its not just a joke it really works more often than I'd like to admit.
~25 years ago I was a firewall administrator, and the Cisco PIX we used was notorious for gradually running out of memory, effectively cutting off the entire company from the Internet when it went down (and it was a very big company).
We absolutely should have scheduled reboots during off hours. Lessons learned.
THIS should be the top comment. So much of our value comes from those bespoke VBA bowls of spaghetti code that get shit done perfectly the same way for the past 20 years that keep the business going. Even the big banks are like this. Someone else said “load bearing excel sheet” that’s pretty good.
I always go with: It is your job to answer. The extended version: If you don't like it you can search for a different job where you won't get questions from me.... but there probably will be others with similar questions or even harder ones. etc basically go on endlessly about the hypothetical them working someplace else. Next time it happens I return to the subject.
Then a coworker showed an even more sophisticated version of the same joke. He created a nearly identical relationship but he would look for job offers matching their resume and pitch them. Look, this is closer to where you live and the pay is better!
I’m going to hope they can figure things out and improve their uptime. It’s so engrained in my teams workflow at this point, tearing it out and switching to anything else would be extremely disruptive.
I did work for a group using Atlassian bitbucket once, and that had its own problems at the time. I’m still pretty sure even a degraded GitHub beats it. That is to say I don’t know what alternatives for GitHub even are
Do any web servers have meaningful updates? It’s pretty standard stuff. IIS can do quite a bit of the same things. It’s common in my industry. We set up URL rewrites, reverse proxys, asp.net, even node.js or python (flask) based apps using WSGI. Separate app pools for everything all on the same server sometimes. It’s never been a bottleneck because it’s pretty streamlined code. Haven't had any security issues with the web server ever. It’s nice there is a GUI but when you start looking at the configuration (xml based) a lot of the directives are familiar if you’re used to nginx.conf or httpd and with htaccess (web.config) in site directories.
The event could be when fair pricing comes from the model providers. We're still at the cash burning stage. When the economy crashes a little and departments start monitoring their spending, and the prices for inference are 10x what they are, there will be less tolerance for employees to substitute constant AI usage for understanding.
reply