Commit graph

107 commits

Author SHA1 Message Date
Jörg Thalheim 87214dbd10
nixos/test-driver: re-introduce log()
Appearantly this is used in tests
2020-08-25 14:50:47 +01:00
Jörg Thalheim f3c0a09c76
nixos/testdriver: sort imports 2020-08-25 10:15:24 +01:00
Jörg Thalheim 392415c285
nixos/test-driver: switch to pythons' logging lib
- Less code
- more thread-safe according to @flokli
2020-08-25 10:13:27 +01:00
Jörg Thalheim c1667f85bb
nixos/test-driver: introduce main method
This way we not accidentally use introduce/use global variables.
Also it explictly mark the code for the mypy type checker.
2020-08-25 09:36:37 +01:00
Janne Heß ff03800d3b
nixos/testing: Fix fail() function
The docs say this behaves as succeed(), but it does not return stdout as
succeed() does. This fixes that behaviour
2020-08-21 21:28:24 +02:00
zowoq 25d7880f17 nixos/lib/*: editorconfig fixes 2020-08-04 00:23:54 +10:00
Justinas Stankevičius f815cb6adf
nixos/test-driver: print a traceback when testScript fails (#92369)
* print a traceback: assertion message can be empty

* change print back to eprint
2020-07-06 23:29:40 +02:00
Michele Guerini Rocco dab676b2d7
Merge pull request #65231 from buckley310/grub-password
grub: add support for passwords
2020-07-01 09:04:30 +02:00
rnhmjoj b520055df6
nixos/lib/test-driver: add wait_for_console_text
This method is similar to wait_for_text but is based on matching
serial console lines instead of the VGA output.
2020-06-24 10:22:53 +02:00
Félix Baylac-Jacqué 7e7aa529d9
test-driver.py: delete VM state directory after test run
Keeping the VM state test across several run sometimes lead to subtle
and hard to spot errors in practice. We delete the VM state which
contains (among other things) the qcow volume.

We also introduce a -K (--keep-vm-state) flag making VM state to
persist after the test run. This flag makes test-driver.py to match
its previous behaviour.
2020-06-21 10:27:51 +02:00
Florian Klink 8a388c8296
Merge pull request #82258 from erikarvstedt/fix-xchg-caching
fix inconsistent caching of VM xchg dirs
2020-06-01 22:34:48 +02:00
Erik Arvstedt d85dc4f690
qemu-vm: fix inconsistent caching of xchg dirs
xchg is advertised as a bidirectional exchange dir, but file content
transfer from host to VM fails due to caching:
If a file is read in the VM and then modified on the host, subsequent
re-reads in the VM can yield old, cached data.
This is caused by the use of 9p's cache=loose mode that is explicitly
meant for read-only mounts.

9p doesn't provide any suitable cache modes, so fix this by disabling
caching.

Also, remove a now unnecessary sync in the test driver.
2020-06-01 21:55:33 +02:00
Erik Arvstedt dfa743e49e
test-driver: remove useless syncs
These syncs have the goal to transfer host filesystem changes to the VM,
but they have no effect because 1) syncing in the VM can't possibly pull
in host data and 2) 9p is accessing the host filesystem on the cached
layer anyways, so even syncing on the host would have no effect in the
VM.
2020-05-27 23:03:15 +02:00
Chuck 751a27020e nixos/test-driver: Specify /bin/sh shell when running a bourne shell script as the user
The test harness provides the commands it wishes to run in Bourne
syntax.  This fails if the user uses a different shell.  For example,
with fish:

  machine.wait_for_unit("graphical-session.target", "alice")

machine # fish: Unsupported use of '='. To run '-u`' with a modified environment, please use 'env XDG_RUNTIME_DIR=/run/user/`id -u`…'
machine # XDG_RUNTIME_DIR=/run/user/`id -u` systemctl --user --no-pager show "graphical-session.target"
machine # ^
machine # [   16.329957] su[1077]: pam_unix(su:session): session closed for user alice
error: retrieving systemctl info for unit "graphical-session.target" under user "alice" failed with exit code 127
2020-05-09 11:01:17 -07:00
Eelco Dolstra 3f80fadec4 testing{-python}.nix: Remove log pretty-printing cruft
This completes the removal of the nested log feature, which previously
got removed from Nix, Hydra, stdenv and GNU Make. In particular, this
means that the output of VM builds no longer contains a copy of
jQuery.
2020-05-07 15:56:30 +02:00
Eelco Dolstra 78f2a83029 test-driver.py: Fix deadlock when the log queue gets full
If a program (e.g. nixos-install) writes more than 1000 lines to
stderr during execute(), then process_serial_output() deadlocks
waiting for the queue to be processed. So use an unbounded queue
instead.

We should probably get rid of the structured log output (log.xml),
since then we don't need the log queue anymore.
2020-05-07 15:25:24 +02:00
Florian Klink 3cdd558a6f nixosTests: drop nr_tests and failed_tests variables
With the tests now bailing out early on a failing subtest, we don't need
to keep a list of failed tests, or the number of total tests
2020-05-01 01:22:42 +02:00
Jacek Galowicz b139bc0d87 nixosTests: Reraise exception in subtests 2020-05-01 01:22:42 +02:00
Silvan Mosberger 85e866db6f
nixos/lib/test-driver: Fix require_unit_state hardcoded formatting 2020-04-06 23:55:04 +02:00
Martin Milata d4cbe042ef test-driver.py: use temporary dir for vde1.ctl
Send SIGTERM instead of SIGKILL to vde_switch to give it chance to
delete the directories.
2020-03-29 00:03:59 +01:00
Rouven Czerwinski 1685cd7fa7 nixos/test: update test-driver.py for mypy 0.770 2020-03-27 01:57:48 -07:00
Jacek Galowicz d772206c49 nixosTests: Make VLAN argument optional in python test driver 2020-02-14 19:05:03 +01:00
Jacek Galowicz 798fcaafa4 nixos/test: Add copy_from_host and copy_from_host_via_shell to python
test driver
2020-02-07 13:24:16 +01:00
Jacek Galowicz f63ef280df nixosTests: Drop invalid utf characters in command output 2020-01-26 17:11:59 +01:00
Florian Klink 764aba4c1b
Merge pull request #77587 from singron/fix-shared
test-driver.py: Share the shared dir between VMs
2020-01-19 22:07:31 +01:00
Eric Culp 6795a4002c test-driver.py: Use /tmp/shared for copy_from_vm
The docstring says it uses a directory shared among all vms, although
that doesn't seem necessary for the functionality. However, it does need
to be consistent between the guest and host.
2020-01-19 10:33:46 -08:00
Florian Klink ed0b5b6133
Merge pull request #77662 from marijanp/verbose-python-test-driver
nixos/test: added verbose output for failed tests
2020-01-17 13:50:49 +01:00
Erik Arvstedt 5bdb653baf test-driver.py: fix decoding of VM output
The codec format 'unicode_escape' was introduced in 52ee102 to handle
undecodable bytes in boot menus.

This made the problem worse as unicode chars outside of iso-8859-1
produce garbled output and valid utf-8 strings (such as "\x" ) trigger
decoding errors.

Fix this by using the default 'utf-8' codec and by explicitly ignoring
decoding errors.
2020-01-14 15:52:44 -08:00
Marijan Petričević 61c61f80e5 nixos/test: added verbose output for failed tests 2020-01-14 09:33:10 +01:00
Eric Culp 21f567bdd9 test-driver.py: Share the shared dir between VMs
This changes the python test driver to match the behavior of the perl
test driver.  I.e. the directory mounted into /tmp/shared should be the
same for all machines.

This probably fixes many tests, but I found this while investigating
failures in nixos/tests/ceph-multi-node.nix.
2020-01-12 15:49:17 -08:00
Jacek Galowicz e05ae69904 nixos/test: Use retry() in all looping functions that need timeouts 2019-12-03 16:59:43 +01:00
Michael Raskin ad38a08ecb test-driver.py: add Machine.copy_from_vm 2019-11-25 11:35:48 +01:00
Michael Raskin ef5bc381f9 test-driver.py: Fix the shared directory reference 2019-11-25 11:33:47 +01:00
Robin Gloster ba03a1b731
nixos/tests: fail on failing subtests
This was the behaviour in the perl test driver too and a regression.

Also cleaned up unused imports
2019-11-25 01:08:06 +01:00
Florian Klink 758efb9348
Merge pull request #73190 from flokli/ceph-tests-python
WIP: nixos/ceph: port tests to python
2019-11-21 20:13:41 +01:00
Jacek Galowicz 4dba4db1db nixos/test: let python driver evaluate test code in global scope so the
user can define and use functions
2019-11-20 19:08:18 +01:00
Jacek Galowicz 72dd1c6a74 nixos/test: If qemu monitor returns 0, simply return (like perl script does) 2019-11-20 17:38:20 +01:00
Jacek Galowicz 67e2042861 nixos/test: Fix child environment of python driver 2019-11-20 17:37:12 +01:00
Jörg Thalheim 03e6ca15e2
test-driver: add mypy support
It's a good idea to expand this in future to test code as well,
so we get type checking there as well.
2019-11-11 13:49:48 +00:00
Florian Klink a8e9a93493 nixos/tests: fix succeed() with multiple commands
we previously immediately returned the first commands output, and didn't
execute any of the other commands.

Now, return the last commands output.
This should be documented in the method docstring.
2019-11-11 13:45:11 +01:00
Leonhard Markert c7becac738
nixos/tests: correct arguments to get_unit_info 2019-11-08 15:01:42 +01:00
Marijan Petricevic 5854aaf680 nixos/tests: improve get_unit_info expressiveness on failure 2019-11-08 11:20:54 +01:00
worldofpeace 1e7ddf233a
Merge pull request #72943 from marijanp/port-test-driver-python
Port remaining test-driver functions to python
2019-11-07 20:23:27 +00:00
worldofpeace 6ed6d1af6a
Merge pull request #72947 from tfc/boot-test-port-fix
Boot test port fix
2019-11-07 17:32:43 +00:00
Jacek Galowicz 52ee1026b0 nixos/test: Handle undecodable bytes
This threw exceptions in boot menus
2019-11-07 10:00:39 +01:00
worldofpeace 3780b9e69c
Merge pull request #72835 from tfc/nixos-integration-test-ports
Nixos integration test ports
2019-11-07 01:05:36 +00:00
Marijan 9915a8ca45 nixos/tests: implemented python test-driver version of forwardPort 2019-11-06 22:43:37 +01:00
Marijan 01f79dfc9e nixos/tests: implemented python test-driver version of dumpTTYContents 2019-11-06 22:43:37 +01:00
Jacek Galowicz cf138b4e6b nixos/tests: Fix subprocess launch call for VM monitor
This crashed in the create-script case
2019-11-06 21:48:08 +01:00
Daniel Schaefer afbfef93a7 nixos/tests: Ignore shutdown/crash if not booted
Condition seems to be inverted. Crash and shutdown only make sense, when
the machine is booted; i.e. we return immediately otherwise.
In the Perl test driver this is:

    return unless $self->{booted};
2019-11-06 16:06:43 +01:00