Wednesday, June 15, 2016

Converting Test to NUnit

So we have been working on converting tests over to NUnit for a couple days now, and we have found a couple different things:

  1. Converting tests to NUnit is not very hard; we had to do some restructuring, but it was no big deal. 
  2. But the tests are not running in parallel like we want them to. We aren't really sure why this is, but we think it has to do with how the code was structured.
  3. We have now decided to scrap everything and start from scratch instead of trying to work with the existing code. 
So now that we have decided to start over and write everything ourselves, its probably going to take at least a month or two. While this means having to redo everything, I'm actually excited about this because it will give me the experience necessary on working on a web-testing framework done from the ground up. 

We are still going to use NUnit to run the tests, and we just have to start working on creating the proof of concept with some small test. Once we can prove that those small tests can run in parallel, we will have the green light to finish the project.

That's all for now, thanks for reading!

Friday, June 10, 2016

Parallel Test Execution

So after doing some research on how to run parallel test using Selenium Grid, I found out a few different things that I hope will be helpful to others:

First, running tests in parallel is actually not up to Visual Studio, Selenium Grid, or the language you are using (Ex: C#). That task falls onto the test runner as long as it supports it. The built in test runner on Visual Studio is called MSTest, and it doesn't support this function automatically. I read on some sites that you are able to change some stuff around to force it, but its not worth it in my opinion.

Second, there are a few different test runners that can run multiple test at once. The ones that I found are NUnit and XUnit for C# and TestNG for Java. I'm sure there are more, but those are the ones that I looked into. There are many differences between NUnit and XUnit, but it looks like NUnit is the one that is most like MSTest and therefore would require the least amount of work in converting the test to be compatible in NUnit.

Third, I had been looking for a while on some tutorials, but since I didn't really know what I was looking for I had a hard time. Once we had settled on NUnit as the test runner for the project we are converting, I was able to find a tutorial series on YouTube that explains in great detail how to run multiple tests at once and even how to run the same test using different browsers.

Now that we know what we are doing, we will be working the next few days on getting a few test converted over and see if we can get it to work properly. That's all for now, thanks for reading!

Tuesday, June 7, 2016

Test Execution Using Selenium Grid

So as I mentioned last time, I have been at work trying to figure out how to get tests to run on Selenium Grid, which wasn't that hard to figure out. The set up just needs a few manual steps, buts its really easy.

First, there is only a few lines of code that we need change in order for it to work. If you remember my last post on Selenium, I will be using the same code I did then and just update it.

The new code looks something like this:

using System;
using System.Diagnostics;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;

namespace SeleniumTest
    class GenerateTokens
        private RemoteWebDriver driver;

        public static void Main()

        private static void Generator()
            var capabilities = new DesiredCapabilities();
            capabilities.SetCapability(CapabilityType.BrowserName, "chrome");
            capabilities.SetCapability(CapabilityType.Platform, new Platform(PlatformType.Windows));
            driver = new RemoteWebDriver(new Uri("http://localhost:4444/wd/hub"), capabilities);

            driver.Url = "";

            for (int counter = 0; counter < 10; counter++)
                // Do something 


As you can see, we use a RemoteWebDriver object instead of just a IWebdriver like we did before. This basically is saying that the test will not be running locally (but it can, more on that later) and that we will be using Selenium Grid to run the test on a different machine. We specify that we want to run the test on a machine that has Chrome as the browser and that its also a Windows machine. After that, we don't really care what computer is running those test, all we care is that it meets those two requirements for the test.

But if you were to run this test now, it would fail, because it wouldn't be able to run the test locally. The test runner is looking for an instance of Selenium Grid at the address that you provided "http://localhost:4444/wd/hub" and it can't find it. Now you have to set up the grid with only one hub and at least one node to run the test. Here are the steps for the set up:
  1. First you have to have Java installed in your machine and make sure that it was added to your PATH. 
  2. Next download "Selenium Standalone Server" from the Selenium downloads page. And put it in a folder that you want, I created a folder in my C driver called Selenium for it. 
  3. You will need to open a command window and navigate to the folder and enter: "java -jar selenium-server-standalone-2.53.0.jar -role hub"
  4. You will then need to open a new command window and navigate to the folder again and enter: "java -jar selenium-server-standalone-2.53.0.jar -role node  -hub http://localhost:4444/grid/register"
At this point you have now set up a grid, comprised of just your computer but it it will get the job done for now. If you open a browser window and go to "http://localhost:4444/grid/console", you will see that your grid has one node available to run tests and which browsers are available on the node.

Now if you run the test that we wrote before, it will work. The test runner will send a test to the server and the server will then decide which "node" to send the test to. Then the server will return the test result to the test runner. That's a really simplistic way of explaining the process, but at least that's how I understand it.

We are still running into issues at work where we can't get the tests to run in parallel, but hopefully we will figure it out soon. That's all for now, thanks for reading!

Friday, June 3, 2016

Baptized by Fire

So I have now been at my new job as a Jr QA Engineer for a little over a week now, and I am no longer as nervous/scared as I was before. But I am still trying to make sure that I understand everything quickly.

For the last week I haven't been doing much, other than getting to know what the automated tests look like that we run on our website. But now I have been given my first assignment, luckily I won't be working on it alone. Another guy started at the company a few days before I joined the team last week, and while he has a ton more experience we are both new to QA Engineering in general, so they decided to pair us up.

We are going to be working on a few different things:

  1. Moving the automated test from an old TFS server and getting it all migrated to GitHub. 
  2. Update the test to make sure they run on Selenium Grid.
  3. Update the tests to that we can run multiple tests at once in parallel using Selenium Grid.
  4. Make the tests run/kick-off from GitHub instead of TFS when a new build gets pushed to the QA environment.
That last one I am not really sure how we are going to do at all, but the other ones I have been researching for the last few days... but there is a problem. All the documentation that I've found is either old, outdated, or for Java and not C# (which is the language which all the test are written in).

I think I am getting close to getting the test to run on Selenium Grid, I am still not sure how to get multiple tests to run at once in parallel. I know that we have to use a testing framework that allows for multi threading, but so far I haven't figured it out.

I will keep looking into it, and when I figure it out I promise to make a tutorial on this site and Youtube. I hate how hard this has been and if I can help someone else, I will do my best.

Anyways, that's all for now and thanks for reading!