#seconf

Minou Sexy à Paris

Blog: https://sladdicted.blogspot.com/2020/01/minou-sexy-paris.html

Flickr: https://flic.kr/p/2i9PcGx

Facebook: https://www.facebook.com/imajica.vemoflow

InstaGram: https://www.instagram.com/imajicavemoflow/

Avatlife: https://www.avatlife.com/Imajicavemoflow

Watch Adam Carmi's talk at SeConf India!

If you don’t know what visual testing is, or if you think that Sikuli is a visual test automation tool, or if you are already automating your visual tests and want to learn more on what else is out there, or if you are on your way to implement Continuous Deployment or just interested in seeing how cool image processing algorithms can be - this talk is for you!

do you do cross browser functional testing

2/3 of crowd runs multiple browser tests, but why?

we can't test rendered things like position and pixel perfect design

and testing for functionality is slow and difficult

functional tests are slow

you need a timely feedback loop or else the devs are forced to do

costly context switching which is painful

you should only run the critical tests in actual browsers in webdriver

js test runner:

karma https://github.com/karma-runner/karma and karma webdriver launcher https://github.com/hindsightsoftware/karma-webdriver-launcher

talks to existing Selenium grid (can reuse existing webdriver infrastructure, inc saucelabs)

node.js

TODO ideas!

run files every time they change, automatically?

starting browsers is super slow, can we do something clever to save time? or re-use?

devs do TDD and use code-coverage tools to assess where QA needs to write additional tests

cucumber?

it's not a magic bullet! gherkin given when then, steps, regexes to match strings, webdriver does things based on match

cucumber != bdd/atdd

homeaway

vacation home service. incorporate companies all from all over and merge front-end/back-end. constantly adding new services.

1000s of test cases per brand

audience

who cares about your test and what is their level of technicality ?

where do you need communication? traditional specs go out of date, devs don't understand 'em, and QA is left in the wind.

putting it in cucumber hopefully means you have living documentation - changes to website need to be reflected in changes to cucumber

cucumber helps focus on the purpose

what do the consumers of the test care about?

mike's password?

how we welcome a user?

content on the page?

exact header text?

test consumers don't care about that stuff! plus it's really brittle!

try and automate things with value - insist on a high bar for automation

cucumber helps improve your naming convention

puts you in a mindset where you name things what they are, and you name them consistently.

because you have to match the regex, the naming conventions can be standardized

cucumber makes it harder to cheat

makes it harder to cheat - not as flexible as RSpec, so you have to be deliberate about your changes

changes need to be business related, who should be involved in making this change?s

"perfect" testing pyramid

so, looking at a new functionality/feature, what kind of test design do we need? long term test coverage? at salesforce scale, UI tests get expensive - resources and time

what if we test without the DOM? can we have the devs write javascript tests?

test without DOM, load scripts and specs into javascript, execute them in the context of different browsers. these can be run much, much faster than webdriver UI tests.

example test

"arrange", "act", "assert" pattern. but it's hard, what to do about dependencies?

mock everything except for what you're testing. don't test things nearby, just test specific things

javascript is application code!

new strategy

bottom: JS unit tests, middle: work with headless DOM like Phantom.js, top: webdriver for ensuring consistent experience and browser compatibility

security

we want to find problems with active and passive scans

if you have selenium covering your entire app, you can use it to do security scans

how: send your selenium traffic through a proxy request and make a queue of requests, then with that list you potentially have all your access points/vulnerabilities

XSS (reflected or stored) (in the server or in the DOM), CSRF

obstacles

CSRF tokens expire on replayed requests (which is working as expected, right? :))

failures and flappers can give a bad queue of requests that source your "entire list" of access points

not timely - you basically run your whole thing twice. once to collect, again to scan security

problem: communication is hard

tough for everyone involved to see what's going on.

solution: cucumber! it's for communication and as a nice side effect it's an automatoin tool

problem: maintanability

solution: page objects

problem: works on my machine

RVM - allows you manage ruby installations. gemsets, bundler

it's a pain getting everything configured on your own. so, use vagrant!

vagrant uses puppet underneath to do all the work

also, jenkins! wahoo.

problem: too many browsers to support!

tough to manage, use saucelabs. it should display in all browsers, but maybe not all the features will work in all browsers.

problem: visiblity

if you have a perfect automation solution but nobody knows about it and no one cares, you're not doing a great job.

try hard to make it visible and open

their jenkins is completely open, failed tests go to saucelabs video

more info at his blog!

Everything is public & transparent, so it's a great resource for examples and problem solutions

questions:

why ruby instead of php: the php bindings sucked big time! also, he was a ruby dev

saucelabs + jenkins integrated for reports?: provider just had a deal with saucelabs, pretty trivial. clone their repo and change some env vars and you can run all the tests on your own computer

how are failing tests connected to saucelabs?: saucelabs uses selenium session id