This project is read-only.

Writing the result sets to disk

Jun 5, 2013 at 11:28 PM
I need to write the result sets from a unit test to disk to make it easier to debug. Is that functionality already supported?

If not, I'm happy to write the code. I'd like your advice on what you think the XML syntax should be and the best place to perform the writing of the data. Your thought?
<test name="vwSSRSMonthlyCounts" uid="0001">
    <system-under-test>
        <execution>
            <query file="queries\vwSSRSMonthlyCounts.mdx" connectionString="@mdx" queryresults="resultsets\vwSSRSMonthlyCounts.mdx.csv" queryresultsformat="csv" />
        </execution>
    </system-under-test>
    <assert>
        <equalTo keys="all-except-last">
            <query file="queries\vwSSRSMonthlyCounts.sql" connectionString="@sql" queryresults="resultsets\vwSSRSMonthlyCounts.sql.csv" queryresultsformat="csv" />
        </equalTo>
    </assert>
</test>
Jun 6, 2013 at 8:17 AM
Edited Dec 26, 2014 at 2:51 PM
I've tried to implement this before but the code had became cumbersome. My problem was that I wanted to persist the resultset only if the test was failing (When the test is successful, I usually don't care about the result-set and I've test suites with hundred or thousand of queries so it can be difficult to find one file in such a repository). It meant I needed to delay the decision to persist or not.

At the end I don't know if exactly this feature was working or not (I don't think so but sometimes I'm a bit lazy just to commit).

Anyway, I've a (draft) branch about this but this branch is old and the merge is not trivial.

The idea was to have something like :
<test name="Log in System-under-test">
        <system-under-test>
            <execution>
                <query name="Select first product" connectionString="Data Source=.;Initial Catalog=NBi.Testing;Integrated Security=True">
                    SELECT TOP 2 * FROM Product;
                </query>
                <log condition="only-if-failed" content="resultSet" file="myFilename" />
                <log content="statistics"/>
            </execution>
        </system-under-test>
        <assert>
            <equalTo>
                <resultSet file="Query.csv"/>
            </equalTo>
        </assert>
    </test>
The log element available in system-under-test and assert.

I'll try to merge tomorrow morning (CET) to give you access to the code base.
Jun 6, 2013 at 6:02 PM
I should be able to implement this around the 1st week of July. Possibly sooner.

Send me a brief description of the options you were planning on supporting.

If you put in a stub function without any functionality and a comment of "// TODO" that will help in pointing me in the right direction.
Jun 7, 2013 at 9:30 AM
I've pushed some basics elements in the new branch "Logs". (Xml and Core)

The vision is to have something generic valid for all assertions and system-under-tests: NBi should be able to log in files the queries executed on the servers, the result sets returned by the servers and some statistics about the time needed to execute the queries or the number of rows returned. It should be possible to configure this at the test level but would be great if we can can use the Default (and config file) also to apply this to the whole test-suite. It should also be possible to specify if you to want log for each execution or only if the execution of the test has failed.

My suggestion for the design would be to have the logs instantiated at the assert/system-under-test and assign them until the QueryEngine. In the query Engine we can use the method SendToLog() to submit information to the log. When we decide if the test is red or green, we can Persist() the logs.

You're welcome to suggest some new features or updates to the design. Always more ideas in two heads than one.
Dec 23, 2014 at 11:03 PM
Greetings all,

I've recently started using NBi for migrating some existing data validation tests. Our current method currently has the result sets written to file which makes it easy to provide exception reporting for our developers. I've been looking at https://nbi.codeplex.com/SourceControl/network/forks/NonExplodingBiped/NBi and through the documentation...Is this functionality actively being worked on? The last commit I can see is June 5, 2013.

Any insight would be greatly appreciated.
Dec 26, 2014 at 2:51 PM
Hello mlantz,

This feature (write the whole content of a result-set to an individual file by test) has been discarded more than once. The main reason is that we don't need it. NBi/NUnit have already some features to persist (part of) the result-sets. If you take a look to the xml file generated by NUnit when running a test-suite, you'll find the first rows of the result-sets (expected/actual) and an analyse of the difference between the two result-sets. I've never met a case where it wasn't enough but I will be pleased to listen if you have a case.
Dec 31, 2014 at 6:09 PM
Edited Dec 31, 2014 at 6:11 PM
Thanks for the prompt response Seddryck.

I'm looking at NBi as a replacement to using BI Quality for our current Unit tests and it would have been a bonus if functionality for asserting a result-set to file by individual test existed or was actively being worked on.

Looking at the XML NUnit generates I'm thinking I should be able create a process to parse out what that information we need to provide the equivalent exception reporting we have in place now.

We have cases where we'd like a CSV list of records if a query surpasses a given amount of rows. I still have to test the row count threshold functionality you've introduced for the 1.9 release but I'm thinking it should be simple enough.

Thanks again and have a happy new year!
Feb 19, 2015 at 2:22 PM
If you want to have the full result of your query in the xml-result, you should take a look to the new feature (1.9): https://nbi.codeplex.com/wikipage?title=Define%20failure%27s%20report%20profile&referringTitle=Documentation
Marked as answer by Seddryck on 2/19/2015 at 6:22 AM