I wish I could set an email rule that automatically replied to any message that contains "do I have to open a ticket?" with a size 20 bolded "YES, YOU ALWAYS HAVE TO OPEN A TICKET. EVERY TIME. ALWAYS."
Plus: Also attach a log of the issue or at least describe it. "Look at the configuration" is NOT a valid "issue" XD
My current mission is getting service desk to include error messages, steps tried, etc in tickets before they escalate to 2nd line support, not too important for me as I can just turn around and ask them but it's a pain for my colleagues in the BVI and EMEA.
I came back from vacation (3 weeks, of which 2 I spent doing some sort of patch up job in my mother's house or mine) to find that all my support tickets now go through another guy before he "escalates" those to me. So now I have literally NOTHING to do the first few hours of the day and just after lunch (I take 30 mins, he takes 2 hours...) and EVERYTHING two hours before clocking out... And the team leader is on vacation for the whole month... XD
Have you ever experienced the thrill of confirm a "DELETE LOCAL DATABASE" command on the most bottlenecking industrial printer of the plant? I did it today. Twice...
No, but I have seen someone take down the largest, busiest high street bank branch in London by clicking "shut down" instead of "log off" when remoted into their server.
Only time I've done something close was when I intentionally rebooted our PBX but didn't realise there were a million updates pending, which took out the phones for the entire morning. Fortunately the blame got put on the sysadmin who'd let the patches build up for so long, and the customer service department bought me a cake to say Thankyou for the quiet morning they had :P
I am so happy i dont work in IT and just a dumb mechanic in produktion wo doenst have to handle customers or even worse other people while doing maintanence.
Way back when i was doing tech support, a coworker and I were fixing a PC, and it was doing some long process that shouldn't be interrupted, honestly forgot exactly what, i think it was some disk data recovery or something. He bumps on the power supply thing with his foot, we both nervously laugh as he says "That was close, imagine if i had bumped into the power switch!". He then proceeds to accidentally bump it again, this time actually hitting the power switch. Panic ensues.
Junior comms engineer good few years ago made an undocumented change to the phone system and knocked out all incoming and outgoing calls at an investment bank. Boy howdy there were some angry bankers that day. He kept his job though, lucky for him he was a decent engineer
Twleve years ago I was working as a software analyst for a company that owned several cemeteries (death is a big business). Despite being a software analyst, I sometimes helped with the server maintenance workloads because I had some knowledge and because the IT support team was short staffed. It was a friday, we were configuring the remote connections to monitor the servers (9 physical) during the weekend because on monday it was independence day and nobody would come to work. So nobody would be left in the office for on premise support for the whole weekend. I logged in a couple of times during the weekend and found no problems, everything was running smoothly. I arrived on tuesday to work to find that all services were gone. All of them, ERP, CRM, mail. Only the fileserver was still available. All the fingers were pointed at us and to our policy of remote monitoring. We were in a hell of a shitstorm. We tried to start troubleshooting, but we couldn't even connect remotely to the servers, and all the status monitors were blank. We had no choice but to get into the datacenter and see if we could get some kind of response from the servers. This is when things started to get weird. First of all, the datacenter was boiling hot and 8 out of 9 servers were roasted. The AC wasn't working, but we couldn't find anything wrong with it. It was just turned off. We decided to leave the detective work for later and focus on recovering whatever we could from the dead servers. Our team spent 72 hours of non stop work to recover all the data, reinstall all the services in other servers (which were older servers used for development and testing) and getting them to work, while new servers arrived. After the shitstorm dropped its intensity, we went back to detective work. We started from the mail server and found a ton of unsent mails. One of the last of these unsent mails went more or less like this: From: asshole@cemeterycompany.com To: Everyone Subject: Energy saving policies Hi all, I came here on monday as I had some documents to review for a customer and I was astonished by the lack of environmental conscience in this company. Lights, air conditioning, computers, everything turned on even though there wasn't a single soul in this office apart from the security guy who kindly opened the office for me. I had to take time from my free day to turn everything off. I hope we can consider the impact of our carelessness on the planet and start enforcing some energy saving policies. Regards, PD: the air conditioning by the IT office was a bit hard to turn off (yes, the one that makes a lot of noise and must consume a ton of electricity). Fortunately, with the help of the security guy, I found the control panel and turned it off. So yeah, security wasn't tight at the office. But we never thought an imbecile would cook more than 50K in servers because he was trying to be eco friendly. We left the message in the queue, updated the unsent queue and the rest is history.
Yes he was fired, the general management sent an apology for blaming IT before we could find the origin of the clusterfuck, and the sales department (from where the frackwit was) sent a memo to all its personnel not to fucking mess with company equipment other than their own.
