Outage/Data Loss

Status
Not open for further replies.
O

Old Cliff

Guest
As most of you know we had a major outage over the last 24/48 hours…. To limit our down time we hired a professional company that provides consulting with relation to MySQL. The company and I have worked diligently all day to bring the server back to a running state. Up to this point the server seems to be running ok and we will be watching the server and making some additional changes over the next few days, but we had to revert to a backup Aug 3rd backup, which means that all posts from Aug 3rd until the server crash on Sunday morning were potentially lost.

Thus far, from our findings, we have concluded that there were some settings that various people suggested in the Vbulletin arena and/or the user communities that with respects to MySQL may have create the problem with the backups and server down time. I am in the process of trying to see if I can recover any or all of the missing data. Please be patient during this time, I assure that we do not like this any more than you do and we sincerely apologize for any inconvenience that this outage and the data loss may have caused you. With the help or our consultants, we should not have endure this problem again!

We will keep you all informed on our progress with data recovery and server reliability.

Cliff
 
Man this server upgrade has been nothing but trouble for you has it? Anyway good luck on your data recovery.
 
hardforum is a computer tech site?? how hard is it to keep the freakin site up and running and not lose dozens of subscribed threads. this is getting pretty ridiculous. :mad:
 
I don't see my GenMay which I paid for. And I lost my [H]lite status, damn.
 
Man, I'm just glad that I retained the e-mails from my recent PM's with contact and tracking information for the guys I was dealing with...:eek:
 
trek554 said:
hardforum is a computer tech site?? how hard is it to keep the freakin site up and running and not lose dozens of subscribed threads. this is getting pretty ridiculous. :mad:
Click to expand...


Get goin for us then. We would love for you to provide us a free website with a ton of bandwith and proper layout like [H]. PM me when you got it going.
 
trek554 said:
hardforum is a computer tech site?? how hard is it to keep the freakin site up and running and not lose dozens of subscribed threads. this is getting pretty ridiculous. :mad:
Click to expand...

dude, if you dont like this forum, then im sure people would be happy if you leave, that would be one less complainer.

shit happens, websites and servers can go down, they are not invincible. stop whining :mad:
 
trek554 said:
hardforum is a computer tech site?? how hard is it to keep the freakin site up and running and not lose dozens of subscribed threads. this is getting pretty ridiculous. :mad:
Click to expand...
Sounds like you want to run a forum of 30,000+ for free off of that PC in your sig there. PM me when you're accepting registrations!
 
trek554 said:
hardforum is a computer tech site?? how hard is it to keep the freakin site up and running and not lose dozens of subscribed threads. this is getting pretty ridiculous. :mad:
Click to expand...


LOL, this isn't some php nuke site with 10 members.
 
DELL_DUDE_B120 said:
dude, if you dont like this forum, then im sure people would be happy if you leave, that would be one less complainer.

shit happens, websites and servers can go down, they are not invincible. stop whining :mad:
Click to expand...

Quickly, trolls unite!

Seriously though, a forum of this magnitude (even more so for offering a pay service) should have a multitude of DR plans.
I don't have the details so I will not point the finger, but something significant must have occurred to warrant a rollback to such an extended date.
 
trek554 said:
hardforum is a computer tech site?? how hard is it to keep the freakin site up and running and not lose dozens of subscribed threads. this is getting pretty ridiculous. :mad:
Click to expand...

If you don't have anything constructive to say please keep your childish, uninformed comments to yourself.

evandena said:
Quickly, trolls unite!

Seriously though, a forum of this magnitude (even more so for offering a pay service) should have a multitude of DR plans.
I don't have the details so I will not point the finger, but something significant must have occurred to warrant a rollback to such an extended date.
Click to expand...

[H} had to shift relocate to a completely new server literally overnight due to circumstances beyond their control. I'd say that's pretty significant. When dealing with a forum of this magnitude there are bound to be issues.
 
no wonder.. I was thinking that you guys were doing a server migration
 
All I have to say is, Cliff, Kyle, and all the peeps at [H] that we appreciate what you do and thanks for the hard work you guys are putting into the forums
 
As long as the [H] exists I will still be here... I don't even know what forum I'd go to besides this. And no I would not want to go to the other genmay...
 
I was about to suggest a sticky explaining the situation, but Cliff's on top of it :)

A quick question--it sounds like all posts from 8/3 and later are toast. Is there zero chance for those posts to be recovered? I ask only because I started a worklog during that time, and with the thread now gone, I'll need to post a new one. It's not a big deal (I had just started, so there wasn't much there yet), but I don't want to end up having duplicate threads if the old thread gets recovered.
 
I love how everyone is always a DR expert, this is one of the largest vbulletin sites in the world and vbulletin on distributed databases is a pain in the ass, I know from bitter experience.

With the forums pushing the boundaries of what the software is capable of and staff from VBulletin actively looking after the forums I'm not sure what else they can do to stop issues when they do come up and saying you should have 'multiple DR plans' is a little silly when they've said the backups haven't worked properly because of recommendations given :rolleyes:

Just one thing thought Cliff, I don't know if its possible easily but if you bumped the thread count and post count up loads past the current numbers then you can probably impex the threads and posts back in when you get a working backup. Just an idea.
 
Yeh I too lost Gen May and my Rank, it sux that this site is having so many issues since its one of my favorite time killers.
 
Big thanks to all you guys who work tirelessly to keep this forum going :cool:

And for the folks who complain about it, I hear Kyle will gladly refund you the cost of admission in times such as these :D
 
Andhar said:
Yeh I too lost Gen May and my Rank, it sux that this site is having so many issues since its one of my favorite time killers.
Click to expand...

Ya it's been bad lately. I think this is just a bump in the road before we have rock stability for months to come. All the people complaining about losing posts are more worried about their e-cock then helpful information for others.
 
trek554 said:
hardforum is a computer tech site?? how hard is it to keep the freakin site up and running and not lose dozens of subscribed threads. this is getting pretty ridiculous. :mad:
Click to expand...

Everyone's ganging up against you since that's the fashionable thing to do with a comment like yours. However, I'll come out in the minority and support what you're saying. However, as FYI, you could've been a little more diplomatic in what you said instead of just venting.

What I find most disturbing is that during the hardship Kyle and his members endured a couple months ago, lessons were apparently not learned. :( In any case, glad to see HF up and running ... for now. The admins' effort is appreciated.
 
I say hurrah to those who've been working to get the site going again :) Obviously I rarely post, but this place does have quite a wealth of info, though it's always taken with a grain of salt of course...
I can't say anything about preventative measures because I don't have the experience there, so I'll keep my yapper shut on the matter :rolleyes:
 
beowulf7 said:
Everyone's ganging up against you since that's the fashionable thing to do with a comment like yours. However, I'll come out in the minority and support what you're saying. However, as FYI, you could've been a little more diplomatic in what you said instead of just venting.

What I find most disturbing is that during the hardship Kyle and his members endured a couple months ago, lessons were apparently not learned. :( In any case, glad to see HF up and running ... for now. The admins' effort is appreciated.
Click to expand...
I completely agree with you. All you brown-nosers can say whatever you want, but the criticism is valid. [H] has been down more time than all other hardware forums I know combined. And yes, the other forums are also free, and no, I won't just leave, I like [H]. You can now go ahead and make a sarcastic comment, or insult me.
 
I will say that with the outages this forum has had, I have long since moved my FS/T dealings to Anand. I like [H] and all, but they just have far better uptime over there.
 
So am I to understand, root cause is SQL changes based on internet forum suggestions... and now you've hired professional consultants to reverse the damage?

Isn't that a bit.... backward?
 
I lost all my inbox contents and about 100 posts in the forums. Will you be able to recover that data ?
 
Iratus said:
I love how everyone is always a DR expert, this is one of the largest vbulletin sites in the world and vbulletin on distributed databases is a pain in the ass, I know from bitter experience.

With the forums pushing the boundaries of what the software is capable of and staff from VBulletin actively looking after the forums I'm not sure what else they can do to stop issues when they do come up and saying you should have 'multiple DR plans' is a little silly when they've said the backups haven't worked properly because of recommendations given :rolleyes:

Just one thing thought Cliff, I don't know if its possible easily but if you bumped the thread count and post count up loads past the current numbers then you can probably impex the threads and posts back in when you get a working backup. Just an idea.
Click to expand...

...it's #176 actually

www.big-boards.com

OT and SA run vBulletin and they are both in the top 10 overall
 
After reading some of the various threads and complaints I wanted to throw out a suggestion/idea I had. I noticed there are a lot of people who complain about the FS/FT thread issues during this kind of thing. Is there any possible way for that section to have some kind of extra redundancy/backups? I'm thinking it's either impossible or really hard, but I just wanted to throw the idea out in the hopes that it might be possible and help some folks out.
 
Stellar said:
...it's #176 actually

www.big-boards.com

OT and SA run vBulletin and they are both in the top 10 overall
Click to expand...

It's picking hairs but SA's not actually a single forum (and is also an incredibly lightweight version of vB) and in 'busyness' [H] is in the 10's afai recall, I remember last year the [H] was in the top few forums anyway.

Don't know if they've done a big clear out of posts but I know it was into the millions years ago, heck Uncamilty was a good portion of that :)
 
Stellar said:
I will say that with the outages this forum has had, I have long since moved my FS/T dealings to Anand. I like [H] and all, but they just have far better uptime over there.
Click to expand...

Meh, Anandtech still has one of the ugliest boards I have seen. Not a good organizational structure either. If you want to talk about an actual competitor, the only one I can think of is XS. I remember once when both XS and [H] were down at the very same time! That was a shitty day at work, haha. But XS was back up and running within the hour.

That said, XS has more polarized posters...
 
Don't care about the rank, but I lost access to GenMay :/ Just recently bought it.
 
Status
Not open for further replies.
Back
Top