Regression Testing
• Mark Eschbach
I’ve finally gotten v2 of my Ipseity project to a functional and deployed state. Now it’s time to begin cleaning up the workspace. I’m always super afraid I’m giong to break things when I begin to clean up so I generally like regression suites for critical paths. For a web application that means selenium!
Interesting directive at the top of the file I asked Bundler to create: # frozen_string_literal: true
. According to Alexis Mas this transforms all string literals within the file to immutable values. They claim it’s faster and provides better string safety. There are quiet a few breaking changes though, including equality.
Alrighty! I’ve got my Gemfile
setup and installing deps. I tend to like Cucumber for this level of tests because it forces focus on user goals within the system. I’m also not trying to achieve 100% coverage either, just ensure critcial paths are working as expected.
# frozen_string_literal: true
source "https://rubygems.org"
gem "selenium-webdriver", "3.0.4"
gem "pry", "0.10.4"
gem "cucumber", "2.4.0"
There is a Ruby Gem called slop
:laughing:. Hmm, I wonder if I can extend Jekyll to use emoji.
I always forget how to get Selenium Webdriver hooked up initiall. Luckily after digging on their site for a minute I was able to find how to do this. Hmm, I’m getting a cryptic error from ChromeDriver about invalid frames. Perhaps my chromedriver is out of date? It’s from a few years ago anyway, probably a good idea to update it. Bingo! Easy to fix there. I’ve created this skelaton a few times now, so I’ve pushed a revision up to Github. Hopefully I remember it’s there!
Alrighty then, on to actual tests! Being an identity management system I have two critical paths: logging in via a shared secret; logging in via a federated ID. The shared secret is the easiest, so I’ll begin with that. I’ll also use that was the initial thread to pull it through the CD pipeline for the project.
Auth via Shared Secrets
Opening the expected landing page in the browser, I quickly realized I’m not sure how to target the selectors when generated via React. With an ID like undefined-undefined-Alias-24990
with that last number changing often I’m not going to be able to easily target it. I could eat the cost of using .uat-{someident}
classes, but I would rather have the tests entirely blackbox. The first search result for testing react with selenium they use this exact technique. Greate minds I suppose. All well, I guess I’ll run with that.
Using the className
attribute on the elements allowed me to define the resulting tags. Now I’ve got a race condition between loading the elements being generated. Should be easy enough to select the element. DOH! Now I’m getting a ‘cannot focus element for sending input. Perhaps I should add the
input` element to the end of the selector? Bingo! Click the button was pretty easy too. Egg-celent!
No waiting for the page to update of course my small client is going to assert an element exists before it’s there. Time to put in a wait. I always forget how to put in waits because it’s different with every language binding for Selenium. A quick search turned up a, you guessed it, StackOverflow post. I created three waiters: an immediate @ 200ms, a fast @ 1s, and a slow @ 2s. Due to using password hashing algorithms I can’t expect an immediate response however I can expect a fast response. From my understanding we humans view a 200ms response as fast (baring video games), with 1 second as quick, and beyond one second as slow. I believe New Relic uses similar rubric in regards to their AppDex scores.
So in typical fashion, I use a driver program for exploring test case building before implementing it in Cucumber. Unforunately as much as I would love to do more right now I’ve got errands to run for finishing out our Christmas obligations. More later! I’m gonna pickle the code and set the pipeline into Jenkins!