Building a regression test suite.

Apr 7, 2014 at 9:22 AM
First, heads up on your NBI project.

Coming from 10+ years of testing in traditional sw development I was somewhat surprised with the lack of automated testing in the BI field (and formalized testing at all)

I'm currently trying to build up a test suite for the DWH and cubes at a new assignment

Is there any way I can run more than 1 test suite in one go without having all the tests in the same nbits suite ?

Basically I want to have different nbits suite and then a master file describing all the tests I want to run. Also having all the test results in same "group" is very messy for reporting.

I made a workaround by using msbuild to run nunit console multiple times with different nunit projects. I then merge all the xml files and convert to html via a nice template with with some rules to map each nunit project into its own category in the html file.

I'm also wondering about the nbi performance of members counting. It seems slow and unable to count over 64k members, so instead I'm using an mdx query to do the counting and then comparing to known result set with range setting like this

SELECT Measures.X ON 0
FROM FinanceDM

Anyway thanks for building this project and making it available to others ;-)
Apr 7, 2014 at 8:10 PM

Currently, there is no built-in support in NBi to run more than one test-suite. Anyway, it's supported by NUnit (see here) and I don't guess any compatibility issue between NBi and NUnit at this level. So it should work.

Improving user-experience, with an easier setup of test-suite and with the management of an hierarchical tree for the tests are definitively on the roadmap but it's also definitively linked to the release of NUnit 3.0 (still no idea for when it will be), because the current trick (or hack) currently used for interfacing NBi and NUnit doesn't support this kind of improvement.

You're correct than you could have an issue with huge dimensions when counting members (or any other tests with members). Your query is surely more efficient than the query currently implemented in NBi. On the other hand, the query currently implemented offers more opportunities to improve the diagnostic if the test is failed. In a next release, I'll try to improve the handling of such case by providing your alternative.

You're welcome, feel free to report any bug or possible areas for improvement.