Make test suite text-independent
goob
Public
Seen by 150
A recent PR to make capitalisation consistent throughout the English UI broke a lot of tests, because the strings expected had slightly different capitalisation. I had already considered the benefit of making these tests independent of the text strings to make them more robust to such trivial changes to text.
I see two ways this needs to be done:
- using I18n.t("path.to.string") syntax instead of plain text for strings;
- using commands such as 'I submit the form' instead of 'I press "Submit"'.
Other types of change may of course become apparent as the work goes on!
Anyway, I've pushed some initial work to my fork so the core devs can have a look at it: here's the commit.
See what you think. Will this be useful work for the project? If so, is my approach the best, or is there a better way to achieve this text-independence?
I've started with the cukes because they're the ones I'm most familiar with, but this can or course be extended to Rspec and Jasmine tests.
I look forward to hearing your views.
Flaburgan Fri 13 Feb 2015 2:15PM
I submit the form
What about pages with multiple forms?
goob Fri 13 Feb 2015 2:38PM
What about pages with multiple forms?
I meant to do that when it is possible. However, it appears to focus on the form which has been filled out in the previous step, as the tests pass; but I agree, someone who understands the syntax would have to check that this will work on pages with more than one form.
Florian Staudacher Mon 16 Feb 2015 12:21AM
strictly speaking steps like "i click 'submit'" should be considered obsolete scaffolding and better be replaced by steps like "i submit the whatever-form" anyway ;)
also, to adhere to common testing best-practices all external dependencies should be stubbed, which would include translations in every other test that doesn't directly test translations.
tl;dr: yes, you would definitely improve test quality with that :)
Jonne Haß Mon 16 Feb 2015 3:42AM
to adhere to common testing best-practices all external dependencies should be stubbed
Our cucumber tests serve as integration tests, so that doesn't apply there.
goob Mon 16 Feb 2015 12:36PM
OK; thanks, both. I'll start on this later in the week if there are no objections from other core members.
tl;dr: yay!
Florian Staudacher Tue 17 Feb 2015 3:45PM
@jhass you're right ... I am still over-stimulated from my software quality course last semester ;)
goob Tue 17 Feb 2015 7:13PM
Just shows that there is a thing as too much quality... :)
goob Sun 8 Mar 2015 2:03PM
I've got most steps working text-independently, but have hit two snags:
- The 'should see the text for' steps don't interpret variables in the strings. For example,
desktop/follows_tags.feature:18looks for the exact string "Follow #%{tag}" and won't accept "Follow #boss". I don't know the syntax well enough to have a hope of solving this. Is there a simple way of accounting for variables in the strings, or should I revert these steps to how they were before? - For some reason, whenever I try to rebase against
develop, only the first commit to this branch is included, so I haven't been able to rebase. I've tried everything I can think of, but with no success. You can see on the commit history that I've been making little commits as I've gone, assuming that I'd be able to rebase into one commit once I was finished. Any help you can give would be much appreciated!
Jonne Haß Sun 8 Mar 2015 2:30PM
I think leave them at the hardcoded text for now and lets revisit when you opened a PR, where it'll be much much easier to discuss solutions.
As for rebasing, I'd need to look over your shoulder to see what's going wrong :/
Jonne Haß · Fri 13 Feb 2015 1:47PM
Yes, I think this is the most pragmatic approach.
I'll leave some comments on your commit.