Anyone else think an open beta is a bad idea? by Imposter - TribalWar Forums
Click Here to find great hosting deals from Branzone.com


Go Back   TribalWar Forums > Current Gaming > Tribes Talk
Reload this Page Anyone else think an open beta is a bad idea?
Page 1 of 4
Thread Tools
Imposter
Tribalwar Staff
Contributor
Old
1 - 05-13-2003, 10:44
Reply With Quote
Ok call me crazy, but I don't want to see an open beta. Here's why-

-The main goal of an open beta vs a closed one is that more people theoretically means more bugs would be discovered. But I think it will not scale that way. You'll get the same bug reported more times.

-You'll get an absolute avalanche of bug reports. Most will be of the same bug but you'll have to categorize them and somehow weed out the duplicates. Theres no way to do that with a computer, it will have to be by hand. You'll probably need someone who understands the code to help categorize them so you know where to start searching for a bug...that takes up a lot of resources.

-I guess a problem I have with it is that an open beta takes away from the surprise of playing a new game. Everyone will want to play a free beta. They'll use that to decide if they want to buy the game. But they'll be playing a crappy, unfinished version of the game. I think you'll lose purchasers of the game because of it.

-It's basically a free trial of the game. So it could generate buzz, but like I said above, buzz isn't always good. See T2.


So what are other options?

-A small closed beta. Downside is you won't find as many errors. Upside is the "surprise" factor of a new game stays there when people first play the game.

-A large (1000+) closed beta. You'll find more bugs but you'll also get leaks where people talk on forums (like this) saying what they think of the game. There's no real way to avoid that. T2 was like this. But for all the errors it found they weren't fixed, which is actually a good point that a beta is only as good as how much time the dev team has to respond. If they waste time with other things instead of fixing bugs then a beta isn't worthwhile.

-Hire a good Q&A team. Find more bugs, get better results, work faster. May miss more obscure hardware related bugs, but that should be rare if the Q&A team is good. Didn't thrax used to do q&a for sierra? I may be mistaken.

So it becomes a cost/benefit analysis. Is paying a good Q&A team more cost effective than a large beta? I think so but I'm obviously not in the industry and am just basing it on intuition. It seems whenever there is an open beta people get tired of the game or talk about it's bugs and don't end up buying the game, ie: Planetside.

Sure players want a free preview of a game before they purchase it. But that can be covered by a seemingly out-dated idea, the demo. Give them 2-3 levels of the single player in a demo and let that generate buzz.

Anyway, I eagerly await the open beta so I can see how the game plays
 
Imposter is offline
 
Sponsored Links
vempire
Resident Freak
Contributor
Old
2 - 05-13-2003, 11:10
Reply With Quote
You do make some good points, but here is something else to think about.

In a closed beta, you will probably be finding a more select group of people. More then likely you would see the biggest names in the Tribes community getting preference for being admitted into the closed beta. After that you would probably see that most of the other people in the beta would be experienced tribes players. This would be good in that they would know how the games have worked inside and out in the past. The problem with that though would be bias and prior knowledge. In any kindof QA/testing environment it is a good idea to get a mixture of people who know what they are doing as well as people who don't. People who know what they are doing will do everything right, which is good because they will find certain bugs. People who don't know what they are doing are also good because they will find other bugs by doing wierd combinations of things.

I worked in software/hardware QA/testing for about a year at the company I work at right now, so I'm quite aware of how to find bugs. We would always do the bulk of the testing on our machines with people who knew how to work them, then we would let random employees here use the machines. You would be amazed at how many issues came up through people doing the wrong things. Then we would go back and kill those bugs. In the end, it makes for a far tighter end product.
 
vempire is offline
 
Imposter
Tribalwar Staff
Contributor
Old
3 - 05-13-2003, 11:16
Reply With Quote
I think you're right in that a lot of "big names" would make it into a small closed beta like in T2. And as you say that has advantages and disadvantages. However they could take that into consideration when they take people into the test.

I'd rather see them hire a Q&A test and do no public beta test.
 
Imposter is offline
 
Void|deadjawa
VeteranX
Old
4 - 05-13-2003, 11:29
Reply With Quote
I think large betas 20000+ that blizzard uses now are the perfect number. That way you get a lot of varied systems played on, but its still manageable.
 
Void|deadjawa is offline
 
Imposter
Tribalwar Staff
Contributor
Old
5 - 05-13-2003, 11:33
Reply With Quote
How is 20000 players managable? How big is blizzard compared to Irrational I wonder? You have to spend a good bit of resources to deal with the input from a large beta.
 
Imposter is offline
 
Kendo
VeteranX
Old
6 - 05-13-2003, 11:43
Reply With Quote
Well cant they just put in like one map for the beta or somthing so people arent totally exposed to the entire game and they will still wanna see what all the other stuff is like, and buy it.

As long as essential things are put in the beta that will test as much of the coding as possible without ruining the surprise would be good to me.
 
Kendo is offline
 
Imposter
Tribalwar Staff
Contributor
Old
7 - 05-13-2003, 11:49
Reply With Quote
If they have a beta with just one map, and leave out things to still keep that "surprise" factor, then they would have missed the entire point of a beta test, because they wouldn't have tested that part of the game they left to be a "surprise".
 
Imposter is offline
 
jsut
VeteranXV
Contributor
Old
8 - 05-13-2003, 11:53
Reply With Quote
They said there would be a very small closed beta before the open beta. "more of an alpha test" was i think how they refered to it.
 
jsut is offline
 
Imposter
Tribalwar Staff
Contributor
Old
9 - 05-13-2003, 11:56
Reply With Quote
T2 had a beta test, that turned into the meta test.

I'm just saying I wouldn't mind if they just hired a q&a team to do the testing and we only got to play the game when it went on sale.
 
Imposter is offline
 
LouCypher
VeteranX
Old
10 - 05-13-2003, 11:58
Reply With Quote
I would hope Sierra and KP both learned from TR2 that closed betas aren't a very good idea.
 
LouCypher is offline
 
Imposter
Tribalwar Staff
Contributor
Old
11 - 05-13-2003, 11:59
Reply With Quote
The "closed" beta of t2 has over 1000 people right? That would fit into my large beta test thing above. It wasn't that they didn't report the bugs, its that the bugs weren't fixed. Having 10000 wouldn't cure that problem.
 
Imposter is offline
 
jsut
VeteranXV
Contributor
Old
12 - 05-13-2003, 12:00
Reply With Quote
From you're own Q

[19:38] <Colosus> *InvalidName* While there has been discussion about having an open beta, will there still be offical beta sign ups for a closed beta as well?
[19:39] <KineticPoet> Maybe not. No, the whole point of an open beta is that its open...there'll be no signups for a closed beta
[19:39] <Thrax> We will have a closed beta before the open beta. It probably won't involve signing up. It will be very small.
[19:40] <Thrax> More like an Alpha test.
[19:40] <Thrax> The true beta will be very open. As open as we can make it.
 
jsut is offline
 
[4M]TheReverend
Unregistered
Old
13 - 05-13-2003, 12:01
Reply With Quote
/agree Vempire

I am looking forward to beta...as long as you don't have to DL it from FilePlanet...
 
 
Imposter
Tribalwar Staff
Contributor
Old
14 - 05-13-2003, 12:01
Reply With Quote
I know what they're plans are man.
 
Imposter is offline
 
Imposter
Tribalwar Staff
Contributor
Old
15 - 05-13-2003, 12:02
Reply With Quote
Why would you want to play in a beta? I'd rather have a demo to test the game with a few weeks before the game comes out. A beta is a demo with bugs.
 
Imposter is offline
 
LouCypher
VeteranX
Old
16 - 05-13-2003, 12:17
Reply With Quote
Quote:
Originally Posted by [4M]TheReverend
/agree Vempire

I am looking forward to beta...as long as you don't have to DL it from FilePlanet...
AMEN!

My preference would be something like BitTorrent.
 
LouCypher is offline
 
SarcaStick
VeteranX
Contributor
Old
17 - 05-13-2003, 12:20
Reply With Quote
I'll **** Thrax if I don't get beta!
 
SarcaStick is offline
 
jsut
VeteranXV
Contributor
Old
18 - 05-13-2003, 12:27
Reply With Quote
I'd rather play in a beta with bugs, than a release version with bugs. I'm sure they could accomplish a non-buggy version of the game with a closed beta, or with hardcore QA, but nothing is going to appease the community until they get to play it.

By having an open beta, assuming the game is stable, and balanced by the end of said beta, sales on release will be better. How many people on TW aren't going to buy the game right away until they get the TW stamp of approval. How much better will initial sales be if that stamp of approval comes during beta, instead of a day or two after release? How much higher will the pre-order's be?

Plus, we have the whole T2 issue to deal with too. How many people on TW think T2 + closed beta = sucks => T3 + closed beta => sucks. An open beta of T3 disassociates the game with T2 even more by not stifling discussion of the game before release. If the game is stable and balanced and fun, nothing could be a better marketing tool than that. If the game sucks, you save your $50, and the game dies a horrible death. However, if the game sucks and there was a closed beta, you might spend you're $50 and be pissed off, and the game will still die a horrible death.

The only thing a demo represents to me is a delay in release. It's another thing to work on. If they do a demo it should be released the day the game goes gold, or close afterwords. I don't think a beta should mean that their is no demo, because beta's tend to not work after release, where as a demo does, and there will be a lot of people that didn't try the beta that would want to play a demo.

Personally I could care less if the beta is open or not as long as i'm on it. It being open means one thing to me, that i don't have to worry about whether or not i'm going to be on it.
 
jsut is offline
 
Last edited by jsut; 05-13-2003 at 12:31..
Imposter
Tribalwar Staff
Contributor
Old
19 - 05-13-2003, 12:55
Reply With Quote
But how many games are actually stable at the end of a beta test? Most seem to leave bugs in to be fixed before shipping...and I have NEVER seen a beta test generate positive news for a game. Take warcraft 3. If there had not been a beta of that game I bet it would have had MUCH higher sales. Instead people played it, didn't like it, and told everyone so. That doesn't mean its not a fun game, it means that some people didn't think it lived up to the hype.

Seriously, an open beta never works for the company that I've seen. I could certainly be wrong though, perhaps someone could provide a good counter-example.
 
Imposter is offline
 
Menzo
Not Sierra Anymore
Old
20 - 05-13-2003, 13:23
Reply With Quote
Quote:
Originally Posted by Imposter
Take warcraft 3. If there had not been a beta of that game I bet it would have had MUCH higher sales. Instead people played it, didn't like it, and told everyone so.
I hate to tell you this, but Warcraft 3 has sold like gangbusters. They shipped over 1.5 million copies to retail on day one alone.

Once you achieve Warcraft 3-style success there's no way you can suggest that they could have sold more if they had done things differently, because there are only three or four PC games ever that have done as well (and most seem to be Blizzard games).

By all accounts, the open beta for Warcraft 3 was a great success - and if we can be half as successful as they were we'll assure the life of Tribes for a long, long time.
 
Menzo is offline
 
Page 1 of 4
Reply


Go Back   TribalWar Forums > Current Gaming > Tribes Talk
Reload this Page Anyone else think an open beta is a bad idea?

Social Website Bullshit


Currently Active Users Viewing This Thread: 1 (0 members and 1 guests)
 
Thread Tools

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


AGENT: claudebot / Y
All times are GMT -4. The time now is 10:15.