Finally - Fast and Real functional on Node checks in Jenkins
Continuous integration and testing is a hot topic. It is so important, it is so good, because it saves us from doing lame stuff ourselfs.
Lago is oVirt's answer to allow fully automated testing with multiple hosts. And it is good at this. Node support has landed a while ago, we just need to get it into production. But for basic Node centric sanity this full blown setup is not needed.
With a simple setup, we are now running sanity tests on a running VM in jenkins.
Our sanity tests now cover the basic building blocks of Node:
Correct bootloader configuration
Thesse items are crucial for the core functionality of node (working upgrades, reliable rollback). And up to now there is nothing oVirt specific, and this is fine, because integration tests in Lago will ensure that the integration is working.
And these tests are isolated (snapshotted), and fast. The complete testing is done in under 2 minutes. The value is high because it will catch at least:
Any partitioning related error
Any severe bug in imgbased
The biggest gap is to ensure that updates are working. More tests can be added, all of this can be driven from python unit-tests.
Let me note that the testing process is robust. The tests are run in real VMs, and are thus not impacted by the host configuration.
The testing is triggered quite easily, in the ovirt-node-ng repository, run:
# Is implicitly called, but this does build the image # 85% of the time make squashfs # This will install the image into a qcow2 # 10% of the time make installed-squashfs # This will check the qcow2 # 5% of the time make check
And this is finally how such a test run looks:
00:23:51.931 The squashfs 'ovirt-node-ng-image.squashfs.img' got installed into the file 'ovirt-node-ng-image.installed.qcow2' 00:23:51.986 ++ sudo -E make check 00:23:51.998 make -C tests check 00:23:52.000 make[1]: Entering directory '/home/jenkins/workspace/ovirt-node-ng_master_build-artifacts-fc22-x86_64/ovirt-node-ng/tests' 00:23:52.004 pyflakes testIntegration.py virt.py agent.py testSanity.py testVirt.py 00:23:52.190 pep8 testIntegration.py virt.py agent.py testSanity.py testVirt.py 00:23:52.481 nosetests --with-xunit -v --no-byte-compile --stop testVirt.py testSanity.py 00:24:43.359 test_agent_works (testVirt.TestNodeTestcase) ... ok 00:24:55.888 test_reboot_works (testVirt.TestNodeTestcase) ... ok 00:25:46.463 test_bootloader (testSanity.TestNode) ... ok 00:25:51.445 test_imgbase (testSanity.TestNode) ... ok 00:25:54.761 test_lvm (testSanity.TestNode) ... ok 00:25:57.731 test_mounts (testSanity.TestNode) ... ok 00:26:00.835 test_packages (testSanity.TestNode) ... ok 00:26:03.741 test_selinux (testSanity.TestNode) ... ok 00:26:06.666 test_services (testSanity.TestNode) ... ok 00:26:06.969 00:26:06.969 ---------------------------------------------------------------------- 00:26:06.970 XML: /home/jenkins/workspace/ovirt-node-ng_master_build-artifacts-fc22-x86_64/ovirt-node-ng/tests/nosetests.xml 00:26:06.970 ---------------------------------------------------------------------- 00:26:06.970 Ran 9 tests in 134.311s 00:26:06.970 00:26:06.970 OK 00:26:06.983 xsltproc nosetests.xslt nosetests.xml 00:26:06.987 00:26:06.987 Xunit Test Results - For: 00:26:06.987 Results for nosetests 00:26:06.987 00:26:06.987 * Tests run: 9 00:26:06.987 * Failures: 0 00:26:06.987 * Skipped: 0 00:26:06.987 * Errors: 0 00:26:06.990 PASSED
The hardest part was to get running correctly in jenkins. But now it does.