Hi,
    I am trying to figure out best testing tool for my project. I have
narrowed down my requirements to two tools NUNIT and VSTS unit. But I have
used neither and I have to use only one of them. Hence can someone who has
used them before share his/her experience on them so that I can get a better
idea and make a proper choice ?
    Any link or suggestions are also welcome.

Thanks and Regards,
Parag
MTS,Persistent Systems Private Limited

They both really do the same thing.  Here are the reasons that I would
choose NUnit over MSs version.

1)  Unit testing in VS can only be done through a high-end version of the
product.  If all your developers do not use the team version, then they will
not be able to use the MS testing framework.
2)  NUnit has been around for years with a proven track records.
3)  NUnit if available for many versions of the framework and also Mono (I
think).
4)  NUnit does not couple the testing to just an MS environment.
5)  External tools may work better with NUnit such as build systems.


Show quoteHide quote

I have used both. nunit is nice, but VS test is suit.  I use it in the VSDev
version.

Show quoteHide quote

They are pretty equivalent. The VSTS tags are a bit closer to the Attributes
throughout the framework, which is nice. If you are using Team System, I
would go ahead and use VSTS. If not, nUnit is a fine alternative. I use both,
depending on the project location.

Show quoteHide quote

There are two things that I really like more about VS.  Better overall
integration with VS.  And you can right-click a method and gen the test stub
automatically.  You also get code coverage results with integration to show
what lines where not covered.  Not sure what nunit offers today in that
regard.

Show quoteHide quote

While NUnit doesnt offer strong vstudio integration, or coverage there
are a number of 3rd party tools that do this integration .... I
personally prefer TestRunner.

That being said, I really believe that the market positioning of VSTS by
MS puts it out of reach for most independent developers and small
development shops.  That being said, IMHO, I personally see NUnit as the
best option.




William Stacey [MVP] wrote:
Show quoteHide quote

Remember you don't need VSTS to use it.  You need at lease VSDev IIRC.  I
have VSDev with the MSDN.  Am I wrong?

I thought same about NUnit before I really used VS.  One day I *forced
myself to use VS for good sized library.  I would not go back now.
Different strokes.  Code coverage is big.

code coverage is available when you use TestRunner and NUnit as well
..... at a fraction of the cost


William Stacey [MVP] wrote:
Show quoteHide quote

"William Stacey [MVP]" wrote

Unfortunatly, I think so.

http://msdn.microsoft.com/vstudio/products/compare/

The Professional Version I often use doesn't have any of the testing stuff,
and unless you got the very, very expensive MSDN subscription, that's the
version of VS.NET that comes with MSDN.

http://ncover.org/site/

William Stacey [MVP] <william.sta***@gmail.com> wrote:
Does it assume a "one test method per concrete method" model though?
That's what Eclipse assumes when you ask it to generate a test skeleton
- and it's almost never what I actually want. (I just start with a bare
test class.)

It's also not useful for true TDD, where the tests come before the code
they're calling.

You can run NUnit over NCover. I haven't used it much myself, but I've
done enough to know it works.

Not 100% sure what you mean.  By default, if you just have a new class, it
will stub out (pretty nice) each method (w/ overload), constructor and
properties IIRC.  And even the private methods if desired.  You can then add
any amount of new test methods, delete or change existing ones.  You can do
any amount of tests (high, low, range, expected exceptions, etc) inside one
method or break them out as you want.  Get nice coverage results, see what
failed.  Click on failed tests to jump to that code.  Click on uncovered
results to jump to that method and see the uncovered lines in color, etc.
Lot more going on here then just red and green traffic lights.

They encourage and support either.  Possibly the easiest way is to stub out
your class with exception in the body, then gen your tests over the class.
Then just keep adding code to your class as you go and iterating your tests.
You could also just stub out your tests without any class first, but not
sure how valuable that would be.  Its all just code, so you can do what you
want.

Have nothing against NUnit.  I can only suggest trying the test and code
coverage in VS (if you got the MSDN w/ VSDev).  I think you will be very
happy with the whole solution.

[I've responded to so many of your posts this last week, I feel like I'm
stalking you]

It's nice - even nicer, to me, is the performance profiler and memory
profiler integration with the VSTS infrastructure. I also love the way you
can rollup smaller tests into big test runs using the Web Stress tools. Very
slick design by Microsoft.

William Stacey [MVP] <william.sta***@gmail.com> wrote:
What *exactly* do you mean by stubbing out? Creating the methods in the
production class, or creating methods in the test class to match the
ones in the production class.

My point is that if I've got a single method in the production class, I
may well want three or four test methods with names indicating what
aspect of that method they'll be testing. Sometimes two methods are
tested by the same set of tests. In other words, creating a "matching"
set of methods in the test code has never seemed that useful to me.

Good to hear VS is finally catching up with Eclipse a bit :)

No, that's not test-driven development. With test-driven development
you don't know what methods you'll have in your class until you've
written your tests.

Yes, but different IDE features can certainly help TDD. I seem to
remember that 2005 is capable of creating methods that you call even
when they don't exist just like Eclipse does. That's what really helps
TDD.

I'm not entirely clear about what "VSDev" is. I'll have to look through
my MSDN discs to find it. I have VS 2005 Pro installed, and that
*doesn't* have the MS test stuff. Have they released the testing stuff
as a free add-in? (I've been gently prodding MS for this for a
while...)

Sorry, was not clear.  I mean either.



Your class:

public class Class1

{

    private string name;

    public Class1(string name)

    {

        this.name = name;

    }

    public string DoSomething(int num)

    {

        throw new NotImplementedException();

    }

}


The default test generated:
....
[TestMethod()]

public void DoSomethingTest()

{

    string name = null; // TODO: Initialize to an appropriate value



    Class1 target = new Class1(name);



    int num = 0; // TODO: Initialize to an appropriate value



    string expected = null;

    string actual;



    actual = target.DoSomething(num);



    Assert.AreEqual(expected, actual, "MessageLib.Class1.DoSomething did not return the expected value.");

    Assert.Inconclusive("Verify the correctness of this test method.");

}

....

You can create your class with just its "interface/signatures" with NotImplementedExceptions until you dev each method.  You could also add methods to your Tests that throw Inconclusive (or what ever) until you implement the methods in the class.

I understand. TMK, it only adds one-for-one by default.  Naturally, in DoSomethingTest() above, you could add multiple "tests" inside that one method and each test would need to pass that particular Assert or you get the respective Assert fails.  You could call any number of helper methods, or create multiple Test methods with a diff name (i.e. DoSomethingTestNull, etc) as you need.  Not sure how a tool would "know" you wanted 2 or 4 test methods for the same overload (or what to call them)?

You could do that above if you wanted.  If class1.DoSomething() did not exist yet, do what you need to do in the DoSomething() Test method.  Are you thinking of something else?

Forgive me.  How would this help you?  There is nothing being tested.  You want that test to fail anyway until you dev the logic.  So why call ghost methods? Just Assert.Inconclusive("Not implemented") in that test method until you hook-up the class logic.  I am surely missing something.

Show quoteHide quoteSorry, all those VS versions confuse me.  Mine is VSTS Software Developer.  With your MSDN pkg, you had to pick if you wanted Developer or Tester - IIRC. You may have it already.  If not, contact your lead and they may be able to change things around for you.  HTH

William Stacey [MVP] wrote:
<snip>

I don't want to write *any* code (not even the signatures) until I've
written a test though. The question is how it makes that.

Eek no - having multiple logical tests within a single test should be
avoided. It makes it harder to work out how much is wrong, because the
first assertion that fails masks later ones which *would* have failed
(unless assertions are handled a very different way in VSTS...)

It can't - which is sort of my point. I can't say I find any of the
skeleton-generators helpful, and can't see how they're likely to
*become* helpful. In other words, whether or not a system includes one
doesn't sway me in favour of it.

I'm not sure whether or not we're talking past each other - see
below...

The point of TDD is that you write the test before you write the method
which implements it - including the signature, often. That way, you
design the API in terms of how to make it usable and testable, rather
than trying to imagine how you *might* want to use it. So, I'd write a
test which called imaginary methods, get the IDE to create dummy
implementations for those methods, and then run the test (which would
fail).

The important thing is that the IDE has to make it easy to generate the
dummy methods given a test which calls them - that's what helps TDD.

Ah, so you *do* need VSTS, as I thought. That's the problem - not
everyone (far from everyone) will have VSTS. If it were available in
Pro but not Express, I'd still view that as a pity - but not being
available in Pro? Bad mistake IMO.

Jon

In effect your writing to some abstract interface that does not exist.
Seems like a waste of time.  May as well at least write the interface first
so you have some idea of what your doing.  That is probably a religious
thing that I should punt on.  If it works for you/others - all the best.

I see your point in general for a large project.  Guess it depends on what
is reasonable for your project and how you want to factor your tests.

ok.  I end up changing the default one too, but find it at least gets me
boot-strapped and off and running on that method.  I understand your point.

If that works for you.  I don't see that saving me any time and would
instead confuse me.  Interface first, for me anyway, seems more reasonable.
Different strokes.

I agree.  The version in Developer *should be included in *all versions.
The extra things like Test Manager could be the value-add in the Test
specific version.  I think they went version *crazy and gave marketing too
much coffee that month.
--wjs

William Stacey [MVP] <william.sta***@gmail.com> wrote:
The point of TDD is that you know the interface which is best to use
when you've used it.

Many others - I suggest you read up on TDD. It's great, although it
does take a certain shift in perspective.

I don't see how the size of the project is relevant - a test should
test one thing and only one thing, where possible.

It's certainly a personal thing.

Indeed. I'm not saying it should do anything without user intervention,
by the way - in Eclipse you click on a lightbulb either in the problem
view or by the side of the line with the compilation error, and that
offers you the option of fixing the problem by creating a new method
(or type, or whatever).

Definitely.

There is nice VS add-in TestDriven.net.
It offer good VS and NCover integration.
It use NCoverExplorer to show code coverage results.

Oleg Zherebin

Show quoteHide quote

[Nunit Vs VSTS]

There are pros and cons to both. The NUnit Stuff is free, and free is
compelling.

The VSTS stuff has integration into Visual Studio. NUnit doesn't. On the
other hand NUnit has some nice VS add-in's that give it integration -
arguably a cleaner integration than VSTS has.

VSTS has Code Coverage metrics, Nunit doesn't. This is a huge win. On the
other hand, NUnit has NCover, which gives you code coverage.

Both Nunit and VSTS have NAnt and MSBuild tasks to run tests and perform
continous integration.

The VSTS test infrastructure is integrated into the Profiler which is very
nice. NUnit has no equilivent that I've been able to find. With Compuware
not offering a Community Edition of their VS 2005 profiler (and the product
being so expensive smaller shops and indiviual consultants can't buy it) ,
this is a compelling win.

The VSTS has the memory profiling stuff built in as well. NUnit has no
equilivent. The SciTech Memory Profiler for $100 is far better than the MS
one, and at a realistic price point.

For 95% of the developers I work with, Team System isn't an option due to
cost. Microsoft's failure to include the Unit Test infrastructure with VS
Pro means almost everyone is stuck with NUnit for testing.

Overall, I like NUnit better - if only I could talk Compuware into giving me
a copy of their DevPartner Pro suite I would be all set.

cool.  Have not tried the memory profiler stuff yet.

Chris Mullins <cmull***@yahoo.com> wrote:

<snip>

That's my feeling as well, although William is talking about this
"VSDev" thing which I don't know about, so maybe it's available to
"normal" MSDN subscribers now.

Personally I think they should have included it even in VC# Express.
They'd have instantly grabbed a *huge* proportion of the unit testing
mindshare, as well as it being good PR in terms of "MS believes in unit
testing" being a good thing to persuade developers of :)

Could someone answer a stupid dgv question?
newbie: why does DataSet AutoIncrement advance by 2?
Search button to filter a Dataset
OutOfMemoryException on DataTables
GridView and reader source