Baseball for the Thinking Fan

Login | Register | Feedback

btf_logo
You are here > Home > Jim's Lab Notes > Discussion
Jim's Lab Notes
— Site News, Baseball Talk, and a Bunch of Other Stuff

Thursday, September 02, 2010

The Great September 1, 2010 Outage

UPDATE: We are back up. Our additional outage was due to DNS problems as our web host didn’t bind the IPs for our DNS servers to the network card on our new motherboard. :(

Whew, we’re back. At about 12:00 PM, I started getting txts and emails about the site being down. I contacted our server host about the outage. It took them a while to diagnose a motherboard failure on our main server. It took them a little while longer to find a replacement board. A long seven hours went by before I saw life. A really long seven hours. We apologize for the downtime.

As frequent visitors have noticed our server configuration is getting a little worn. Along with this off-season’s major site redesign we’ll also be upgrading our network and hardware. I hope you can bear with us until then.

As always…thanks for your patience and support.

Jim

Jim Furtado Posted: September 02, 2010 at 12:17 AM | 88 comment(s) Login to Bookmark
  Related News:

Reader Comments and Retorts

Go to end of page

Statements posted here are those of our readers and do not represent the BaseballThinkFactory. Names are provided by the poster and are not verified. We ask that posters follow our submission policy. Please report any inappropriate comments.

   1. Pat Rapper's Delight Posted: September 02, 2010 at 02:45 AM (#3631763)
The Curse of Francoeur
   2. Dale Sams Posted: September 02, 2010 at 11:52 AM (#3631764)
This motherboard...it was made in Boston, yes?
   3. Infinite Joost (Voxter) Posted: September 02, 2010 at 12:15 PM (#3631767)
I'm just happy it's back.
   4. AROM Posted: September 02, 2010 at 12:16 PM (#3631769)
Boston is in the U.S., so my guess would be not. Americans aren't supposed to make stuff, just buy stuff.
   5. cercopithecus aethiops Posted: September 02, 2010 at 12:28 PM (#3631774)
This motherboard...it was made in Boston, yes?

No. But the (mis)diagnosis was done there.
   6. Ray (RDP) Posted: September 02, 2010 at 12:35 PM (#3631777)
It was the sixth best motherboard in the country.
   7. Jose Can Still Seabiscuit Posted: September 02, 2010 at 12:42 PM (#3631779)
My boss got more productivity out of me yesterday than he has in several months.
   8. Joe Bivens, Minor Genius Posted: September 02, 2010 at 12:45 PM (#3631782)
Never saw the motherboard fail.
   9. Morty Causa Posted: September 02, 2010 at 12:57 PM (#3631786)
That motherboard was this close to retirement.
   10. BDC Posted: September 02, 2010 at 01:03 PM (#3631791)
And I can't understand it, because this spring, the motherboard was in the best shape of its life.
   11. Yeaarrgghhhh Posted: September 02, 2010 at 01:09 PM (#3631795)
Rob Dibble thinks the motherboard should have toughed it out.
   12. AndrewJ Posted: September 02, 2010 at 01:17 PM (#3631805)
Pass.
   13. Rickey! trades in sheep and threats Posted: September 02, 2010 at 01:17 PM (#3631806)
I'm awarding the points to DiPerna @ 6.

The upside of the outage is that the LaRussa/Beck threads seem to have wilted on the vine.
   14. Spahn Insane Posted: September 02, 2010 at 01:19 PM (#3631809)
I don't understand; that motherboard was in the best shape of its life.
   15. rr Posted: September 02, 2010 at 01:34 PM (#3631818)
So's your motherboard.

Never saw the motherboard.
   16. Tuque Posted: September 02, 2010 at 01:35 PM (#3631819)
That motherboard is dumber than ten dogs.
   17. Guapo Posted: September 02, 2010 at 01:43 PM (#3631828)
By any standard that motherboard was one of a small number of pieces of hardware to work successfully inside the main server, it played most of the roles one can play and holy god it's sick of listening to you act as if you've had 1% of the success the motherboards you criticize have had.
   18. Fly should without a doubt be number !!!!! Posted: September 02, 2010 at 01:44 PM (#3631829)
Once again, this is why we virtualize our servers.
   19. AndrewJ Posted: September 02, 2010 at 02:04 PM (#3631842)
It's my motherboard! (slap)

It's my sisterboard! (slap)

It's my motherboard! (slap)

It's my sisterboard! (slap)
   20. David Nieporent (now, with children) Posted: September 02, 2010 at 02:06 PM (#3631843)
That's what she said.
   21. My Grate Friend Peason's pants are rankled Posted: September 02, 2010 at 02:15 PM (#3631848)
It's Manny's fault.
   22. SoSH U at work Posted: September 02, 2010 at 02:15 PM (#3631849)
THe motherboard should have its childboards taken away from it.
   23. hokieneer Posted: September 02, 2010 at 02:16 PM (#3631850)
But it's the motherboard's ethic duty to society to have childboards.
   24. Shooty Survived the Shutdown of '14! Posted: September 02, 2010 at 02:19 PM (#3631854)
So..did any of you guys have to interact with real people, too? I talked to a guy named "Steve" and one name "Jonathan". Ridiculous!
   25. villageidiom Posted: September 02, 2010 at 02:32 PM (#3631872)
Joe Morgan said that Jim Furtado should never have created that motherboard.
   26. Gamingboy Posted: September 02, 2010 at 02:42 PM (#3631887)
Nyjer Morgan was so distraught by the lack of Think-Factory that he took his anger out on Volstad.
   27. Kurt Posted: September 02, 2010 at 02:44 PM (#3631891)
What did Tommy Lasorda think of the motherboard's performance?
   28. Baldrick Posted: September 02, 2010 at 02:44 PM (#3631890)
The motherboard is the district attorney!
   29. Kurt Posted: September 02, 2010 at 02:45 PM (#3631893)
Dave Cameron wants to trade the motherboard for John Danks.
   30. A big pile of nonsense (gef the talking mongoose) Posted: September 02, 2010 at 02:55 PM (#3631911)
I'm sick & tired of players like Albert Belle & Chris Truby not binding the IPs for DNS servers to network cards on new motherboards.
   31. My name is Votto, and I love to get blotto Posted: September 02, 2010 at 02:56 PM (#3631912)
James Andrews was too busy with Strasburg, I guess.
   32. Dale Sams Posted: September 02, 2010 at 02:58 PM (#3631917)
The motherboard is the district attorney!

All the motherboards on the train did it.
   33. Gary Truth Serum Posted: September 02, 2010 at 03:01 PM (#3631924)
That motherboard would have worked just fine if Obama weren't such a shitty president.
   34. Bourbon Samurai in Asia Posted: September 02, 2010 at 03:02 PM (#3631927)
I told that motherboard it was a trap.
   35. bobm Posted: September 02, 2010 at 03:06 PM (#3631932)
The mosque thread hijacked the motherboard. :)
   36. Randy Jones Posted: September 02, 2010 at 03:13 PM (#3631937)
That motherboard was a fraud that pooped its pants.
   37. RoyalsRetro (AG#1F) Posted: September 02, 2010 at 03:20 PM (#3631951)
What does the motherboard have to do with Frank Tanana?
   38. steagles Posted: September 02, 2010 at 03:46 PM (#3631996)
that's the thing about this motherboard. it's not afraid to blow up and knock the site offline for a day. you've gotta admire it for that.
   39. Daunte Vicknabbit! Posted: September 02, 2010 at 03:50 PM (#3632005)
The motherboard needed more public funding to effectively compete.
   40. Yeaarrgghhhh Posted: September 02, 2010 at 03:55 PM (#3632020)
Baseball Prospectus's motherboard would listen if Jim called.
   41. Kurt Posted: September 02, 2010 at 03:57 PM (#3632030)
Daily News: Jim Furtado plays golf while motherboard likely buried with documen
   42. Craig in MN Posted: September 02, 2010 at 04:00 PM (#3632035)
Operation shutdown!
   43. Dag Nabbit is part of the zombie horde Posted: September 02, 2010 at 04:08 PM (#3632049)
I contacted our server host about the outage. It took them a while to diagnose a motherboard failure on our main server.

Daly oughta adopt "our main server" as his new site name.
   44. Dale Sams Posted: September 02, 2010 at 04:12 PM (#3632060)
Compressed air is the new market efficiency.
   45. chemdoc Posted: September 02, 2010 at 04:23 PM (#3632081)
The motherboard eats its young.
   46. WhoWantsTeixeiraDessert Posted: September 02, 2010 at 04:23 PM (#3632082)
That motherboard appeared to be performing suspiciously better as it got older before it broke down.
   47. Textbook Editor Posted: September 02, 2010 at 04:43 PM (#3632103)
If Larry Bowa doesn't buy the motherboard that bottle of Bushmills, none of this would have happened.
   48. phredbird Posted: September 02, 2010 at 05:05 PM (#3632134)
*throws motherboard*
   49. Random Transaction Generator Posted: September 02, 2010 at 05:06 PM (#3632136)
The motherboard's WAR (web-pages-served above replacement) showed that it was under-performing.
Its DIPS (digital interaction processing score) value also indicated it was due for a correction and crash.
   50. Bourbon Samurai in Asia Posted: September 02, 2010 at 05:06 PM (#3632137)
Behold!
Operation shutdown!

the winner!
   51. Edmundo got dem ol' Kozma blues again mama Posted: September 02, 2010 at 05:06 PM (#3632138)
Terrific performance here guys. The Primey goes to ...

gef, for #30.


Those IPs need to get out of the their motherboard's basement once in a while.
   52. Baldrick Posted: September 02, 2010 at 06:16 PM (#3632216)
Operation Shutdown is pretty excellent. #30 is also great. #41 doesn't have the long-term pedigree but is nicely topical.
   53. villageidiom Posted: September 02, 2010 at 06:42 PM (#3632236)
If Larry Bowa doesn't buy the motherboard that bottle of Bushmills, none of this would have happened.
Detectives believe alcohol was involved.
   54. Los Angeles El Hombre of Anaheim Posted: September 02, 2010 at 07:06 PM (#3632259)
TINSTAAM
   55. Guapo Posted: September 02, 2010 at 07:16 PM (#3632272)
We never should have put Grady Little in charge of deciding when to replace the motherboard.
   56. Petunia inquires about ponies Posted: September 02, 2010 at 07:21 PM (#3632279)
You people are ridiculous. You sit there with your spreadsheets and criticize this motherboard for not being a superstar - have YOU ever even tried to bind IPs for DNS servers to a network card at a professional level? Until you've been there, and done what that motherboard has done, you need to shut the #### up.
   57. The Ghost's Tryin' to Reason with Hurricane Season Posted: September 02, 2010 at 07:29 PM (#3632289)
Jim, thanks for enabling our behavior.

As soon as I quit laughing, I am hitting the Donate button on the upper left corner and putting in some money to help Jim fund the upgrade. I invite company.
   58. Edmundo got dem ol' Kozma blues again mama Posted: September 02, 2010 at 07:35 PM (#3632296)
I am hitting the Donate button

What happened to the little contribution diamond next to Donater's names? Or am I behind on my payments (could very well be).
   59. Jim Furtado Posted: September 02, 2010 at 07:37 PM (#3632301)
Very, very funny...now. It wasn't so funny when the site was down and I was anxiously awaiting info about why the site was down. :)
   60. The Ghost's Tryin' to Reason with Hurricane Season Posted: September 02, 2010 at 07:53 PM (#3632328)
What happened to the little contribution diamond next to Donater's names? Or am I behind on my payments (could very well be).


It looks to me like threads in Jim's Lab Notes don't get the diamonds.
   61. Andere Richtingen Posted: September 02, 2010 at 08:21 PM (#3632350)
While the new motherboard has got things up and running, performance is still pretty bad. I have had a lot of pages hang for a long time today. It was the same in weeks prior to the big crash.
   62. Non-Youkilidian Geometry Posted: September 03, 2010 at 12:36 PM (#3632733)
You all should get out of your motherboards' basement and go watch a web host bind the IPs for a DNS server to a network card.
   63. villageidiom Posted: September 03, 2010 at 12:51 PM (#3632739)
Nomar would've saved the motherboard.
   64. Jim Furtado Posted: September 03, 2010 at 12:57 PM (#3632745)
Our server setup is dated. Other than the recent problem with the web server's motherboard, it's running fine. The database server, on the other hand, is on its last legs. My plan is to update the DB to a much faster server with a lot more memory as soon as the season is over.
   65. Infinite Joost (Voxter) Posted: September 03, 2010 at 01:20 PM (#3632758)
Billy Beane should never have built that motherboard.
   66. Devin has a deep burning passion for fuzzy socks Posted: September 03, 2010 at 01:30 PM (#3632769)
David Samson was supposed to replace the motherboard, but it was on the top shelf. [/Brattain]
   67. WillYoung Posted: September 03, 2010 at 01:37 PM (#3632777)
It's ok, Jim. Hawk Harrelson already apologized to Frank Thomas about it.
   68. Fly should without a doubt be number !!!!! Posted: September 05, 2010 at 01:38 AM (#3633645)
Jim, you might consider putting the DB onto a virtualized server sooner, rather than later. The site has been more-or-less unusably slow today, and it's only going to get worse before the off season. There's plenty of cheap virtualization solutions available out there, and they'd certainly offer a decent stopgap until you can get real hardware (although, after the Great Motherboard Fiasco, it's hard to make an argument in favor of owning hardware or your current colocation service.)
   69. Andere Richtingen Posted: September 05, 2010 at 01:45 AM (#3633647)
Our server setup is dated. Other than the recent problem with the web server's motherboard, it's running fine. The database server, on the other hand, is on its last legs. My plan is to update the DB to a much faster server with a lot more memory as soon as the season is over.

I truly appreciate all you're doing, but the situation seems more critical than that. Here it is a Saturday in early September, and I'm getting database failure notices and page load timeouts. If these problems are at all traffic-dependent, I don't see how the site is going to be usable in October. And if they aren't traffic-dependent, they are still quite serious. It ain't my time and it ain't my money, but I really think this is a critical problem.
   70. Lassus Posted: September 05, 2010 at 01:58 AM (#3633649)
The Great September 1 2010 Outage
Not to pile on, but I must agree with #69.
   71. Accent Shallow Posted: September 05, 2010 at 02:03 AM (#3633650)
Thirded. And to put my money where my mouth is, I will donate tomorrow. I can scan cash and send you guys an image, right?
   72. GGC don't think it can get longer than a novella Posted: September 05, 2010 at 02:50 AM (#3633669)
So..did any of you guys have to interact with real people, too? I talked to a guy named "Steve" and one name "Jonathan". Ridiculous!


I've been having to do that a lot this year now that my company decided to block sports sites. It's a weird filter, I can still read Aaron Gleeman and some newspaper's sports pages. I can also read Football Outsiders through aaronschatz.com, but it's not the same as Primer and I cannot comment. And our server blocks jimfurtado.com.
   73. Gold Star - just Gold Star Posted: September 05, 2010 at 03:10 AM (#3633690)
It ain't my time and it ain't my money, but I really think this is a critical problem.
What worries me is, we've had these problems before. We've talked about what we'd like in a site redesign. We've vowed to donate money when the time comes for an upgrade.
Yet, nothing has happened.
   74. Jolly Old St. Nick Is A Jolly Old St. Crip Posted: September 05, 2010 at 03:16 AM (#3633700)
Okay, I just re-upped. Let's put our Jacksons where our jawbones are.
   75. Jim Furtado Posted: September 06, 2010 at 03:13 PM (#3634313)
Thanks for the donations. I appreciate it very much. With the off-season update, I'll need to spend cash on the servers and I plan on getting some help on the templates from a professional web designer.

This update will be a major undertaking because Expression Engine recently completed a new version which requires the re-writing of scripts. Add in the need for a mobile version and the addition of a bunch of new features and I'll be quite busy this off-season. With some of my mistakes in the last update, I'm spending a lot more time thinking through the way things fit, especially the integration of some social media stuff.

Again, thank you.
   76. Fly should without a doubt be number !!!!! Posted: November 30, 2010 at 03:57 AM (#3698999)
If you have paid your webhost even 1 penny in the last two years, they should be charged with theft.
   77. Jolly Old St. Nick Is A Jolly Old St. Crip Posted: November 30, 2010 at 04:47 AM (#3699025)
Has anyone else noted that the pages are taking longer to load these days? I haven't had this problem on any other website, including BB-ref, which I go to at least a dozen times a day.
   78. Petunia inquires about ponies Posted: November 30, 2010 at 05:12 AM (#3699034)
Data being the plural of anecdote, I would agree with [77].
   79. Jolly Old St. Nick Is A Jolly Old St. Crip Posted: November 30, 2010 at 05:48 AM (#3699044)
I just assume it's a conspiracy against Yankee fans who dare speak out against the hegemonic Red Sox clique that rules BTF.
   80. cardsfanboy Posted: November 30, 2010 at 05:53 AM (#3699048)
For a couple of hours today the load time was horrible, for about the last hour or so it seems to be working just fine.
   81. Petunia inquires about ponies Posted: November 30, 2010 at 06:11 AM (#3699055)
I just assume it's a conspiracy against Yankee fans who dare speak out against the hegemonic Red Sox clique that rules BTF.

That'll be the last time I ever agree with you.
   82. smileyy Posted: November 30, 2010 at 06:32 AM (#3699061)
I always figure that its the NBA Playoffs thread bringing the server down.
   83. Lassus Posted: November 30, 2010 at 06:59 AM (#3699069)
I'd like to take this moment to agree with #73.
   84. Harold can be a fun sponge Posted: November 30, 2010 at 07:44 AM (#3699074)
Thanks for the donations. I appreciate it very much. With the off-season update, I'll need to spend cash on the servers and I plan on getting some help on the templates from a professional web designer.

Jim, it seems like BTF has had a lot of downtime due to hardware failures (the motherboard this time, and others in the past), and mis-configuration on the part of your hosting service (the DNS issue this time, and others in the past).

Have you considered moving to some kind of virtualized hosting? This way, if there is a hardware failure, the host can be brought up on different hardware. Have you considered moving to a different hosting provider?
   85. Harold can be a fun sponge Posted: November 30, 2010 at 07:56 AM (#3699075)
I'm spending a lot more time thinking through the way things fit, especially the integration of some social media stuff.

What do you have in mind there? Do users want that?

I think, say, giving users the option of using Facebook Connect rather than creating an account could be quite useful for adding new users (specifically, people who happen upon the site for the first time or who lurk without an account could post more easily). But other types of social media integration probably take a backseat to stability, performance, and usability.
   86. Jim Furtado Posted: December 01, 2010 at 01:12 PM (#3699796)
I'm looking into some of the recent outages. I haven't had time in the last few weeks due to my schedule. I have some time set aside over the weekend.

I'm working on the server issue. My plan is to switch over to new servers some time during February, shortly before I expect to have the redesign done. Our servers are getting old and obsolete. I fully expect the change will greatly improve performance. Rather than tweaking the underlying set-up, I'm redoing the underlying code in Expression Engine's 2.1 code base, with a much simplified structure. I'm also investigating a much better way to archive our old content.

Vinay, as for the social networking stuff, I'll be integrating Facebook Connect, some other Facebook and Twitter stuff, and also introducing a friends/group module. The friends/group stuff will make it easier for members to find what their friends are viewing discussing, which is probably the most popular feature request (after improved performance).
   87. The Primer Chimp Posted: January 05, 2012 at 09:39 PM (#4029973)
This is a test post.
   88. The Primer Chimp Posted: January 05, 2012 at 10:08 PM (#4029985)
Another test. Work?

You must be Registered and Logged In to post comments.

 

 

<< Back to main

BBTF Partner

Support BBTF

donate

Thanks to
Andere Richtingen
for his generous support.

Bookmarks

You must be logged in to view your Bookmarks.

Syndicate

Page rendered in 0.7361 seconds
67 querie(s) executed