Stylecop and NUnit revisited

Couple of points from the previous post:

"At the moment, I still need to install my rules.dll on the installation folder of StyleCop, otherwise new rules will not appear on the settings. I have the suspicion that is because the StyleCop editor uses the dlls that are located on the same folder. I will have to test that assumption."

The assumption was correct. Good, as I don't have to close Visual Studio when wanting to test a new rule.

"At the moment I am using the output path as the location of the source code/dll/settings. I am trying to set a specific folder that will not depend on where you have the solution (using pre/post build events to make sure that it exists and has all the information), but then NUnit on Resharper throws a tantrum that I have the same dll in two different places (one that specific folder, the other the temp folder). Not sure why it doesn't fail when the location is the output path."


It doesn't fall down when used directly from NUnit (instead of using the Resharper ability inside Visual Studio). So will not pursue a solution for the time being (though probably will not be long when I have to do it).

Basic unit testing of Stylecop with NUnit

The Set up
As part of the changes I have managed to introduce on my department, we have a specific coding guideline and we use StyleCop to enforce it. Actually, until now enforce it only in part, as we did not have time to set up the new rules that we have agreed on.  Now I have the time to set them up correctly.

Of course, first one that I created didn't work. And it was time to bring on the testing (which I should have done from the beginning, still coming to grips with TDD). And Oh Boy!, was it complicated. There is not much explanation of what you need to do and the two examples that I located (Stylecop source code, and Stylecop Contrib are far more complicated that what I wanted. Especially as they seemed to use files for testing that could have more than one violation, and then they filter the violations to see if the one they are looking for is there.

What I want? Easy, to be able to test a single rule (nothing else).

The Solution (basic approach)

The basic code that I created is set below (look further down for an explanation). Of course, that is the most basic, and as soon as you have two test methods, you will want to extract the code.

    [Test()]
    public void Test()
    {
      String filePath = LocationOfCodeUsedForTesting;
      String styleCopSettings = Path.Combine(LocationOfDlls, "StyleSettings.StyleCop");
      StyleCopConsole console = new StyleCopConsole(styleCopSettings, false,null, new List()
      {
        LocationOfDlls
      }
    , true);

      CodeProject project = new CodeProject(Guid.NewGuid().GetHashCode(), LocationOfDlls, new Configuration(new String[0]));

      Boolean result = console.Core.Environment.AddSourceCode(project, filePath, null); 

      List violations = new List();
      console.ViolationEncountered += (sender, args) => violations.Add(args.Violation);
      console.Start(new List() { project }, true);

      violations.Count.Should().Be(1);

      violations[0].Line.Should().Be(16); 

 }

Explanation
First, the filepath is for the code that we want to test. LocationOfCodeUsedForTesting is just a variable with the location.

StyleCop settings are the settings that we want to use. If we pass null on the following line, it will use the default installation.  Personally I have set up only active the rules that I am creating, and then I select to not use the parent settings.

The StyleCopConsole is the object that will analyze the data. The first parameter is the settings that we want to use (as said before, null for the ones present on the installation of StyleCop). The null represents the output path. If we specify an actual path, it will try to write output there (but for testing, I wasn't interested). The fourth one represents the location of the dlls for StyleCop and our own rules.

The CodeProject object will represent a project. Not sure the second parameter is needed (and cannot test it now). The first one is a unique identifier, using a Guid because is just a throw away identifier just to run test.

It is a funny way to add a source code to a project. We call the environment inside the StyleCopConsole and pass the project to which we want to add the source code and the location of the source code.

We then need to hook to the ViolationEncountered event, so we can get a list of violations.

And then we analyze the code (console.start).

To do the checks, I am using FluentAssertions for NUnit.

Facts and next steps
There are a few things that I found interesting/baffling/in need of research:

Probably, because I am using Resharper, the executing assembly when I run/debug the tests is in a temporal folder. I cannot use that temporal folder to store the source code/dll/settings. For some reason it fails (I think there could be a problem of permission).

I couldn't use the default location of the stylecop.dll (which is where you have to put your own rules.dll as well), because that requires that every change I make I have to close Visual Studio and reopen it.

At the moment I am using the output path as the location of the source code/dll/settings. I am trying to set a specific folder that will not depend on where you have the solution (using pre/post build events to make sure that it exists and has all the information), but then NUnit on Resharper throws a tantrum that I have the same dll in two different places (one that specific folder, the other the temp folder). Not sure why it doesn't fail when the location is the output path.

The files that I use for testing have a Build action of "None" and have copy to the output folder if newer. While running StyleCop inside Visual Studio it will not analyze files that do not have the Build action set to "Compile". Which puzzled me for a few seconds (why my tests pass, but then it doesn't actually found anything on the file while running normally?)

At the moment, I still need to install my rules.dll on the installation folder of StyleCop, otherwise new rules will not appear on the settings. I have the suspicion that is because the StyleCop editor uses the dlls that are located on the same folder. I will have to test that assumption.

Conclusion
Still a few things to be done, but the basics for setting up testing of StyleCop rules are there. I like the approach more than the other two mentioned (StyleCop and StyleCop Contrib), because I prefer to have a single file with a single issue that is easy to test. Unit tests should only test a single thing without noise.

Well, hope the information is helpful for you. I will add some information in the future when I solve the couple of stumbling blocks that I have.