Differences

This shows you the differences between two versions of the page.

--- testsuite:developerhowto [2014/10/14 00:21] – Provide pointers to good working examples integrating MK_TESTS != "no" in the parent directory EnjiCooper
+++ testsuite:developerhowto [2023/01/06 12:45] (current) – Add ATF-python details AlexanderChernikov
@@ Line 3: / Line 3: @@
 Before reading this page, make sure to read ''%%tests(7)%%'' and [[http://svnweb.freebsd.org/base/head/share/mk/bsd.README?view=markup&pathrev=HEAD|src/share/mk/bsd.README]] for details.
-This document is based on a [[http://wiki.netbsd.org/tutorials/atf/|similar tutorial in NetBSD]] and a lot of text has been copied form there.
+This document is based on a [[http://wiki.netbsd.org/tutorials/atf/|similar tutorial in NetBSD]] and a lot of text has been copied from there.
 ====== Adding a new test case ======
@@ Line 23: / Line 23: @@
   - Choose a sane name for the test program.  **Avoid naming the test program the same as your test case!**  If you find yourself in this situation, it probably means that you are choosing a bad name for either of them.
-  - Create the test program source file.  Refer to ''%%src/share/examples/tests/%%'' for test program sample code.
+  - Create the test program source file.  Refer to ''%%share/examples/tests/%%'' for test program sample code.
   - Add the new test program to the existing ''%%Makefile%%''.
@@ Line 33: / Line 33: @@
   - Do the same as above.
-  - Create the ''%%Makefile%%'' for the directory.  Refer to ''%%src/share/examples/tests/%%'' for ''%%Makefile%%'' sample code.
+  - Create the ''%%Makefile%%'' for the directory.  Refer to ''%%share/examples/tests/%%'' for ''%%Makefile%%'' sample code.
   - Edit the parent ''%%Makefile%%'' to recurse into the new subdirectory.
-  - Edit ''%%src/etc/mtree/BSD.tests.dist%%'' to register the new subdirectory.  Note that if you are adding tests to, say, ''%%src/usr.bin/du/tests/%%'', the directory you register in the mtree file should be ''%%/usr/tests/usr.bin/du/%%''; i.e. the layout under ''%%/usr/tests/%%'' must match the layout of ''%%/usr/src%%''.
+  - Edit ''%%etc/mtree/BSD.tests.dist%%'' to register the new subdirectory.  Note that if you are adding tests to, say, ''%%usr.bin/du/tests/%%'', the directory you register in the mtree file should be ''%%/usr/tests/usr.bin/du/%%''; i.e. the layout under ''%%/usr/tests/%%'' must match the layout of ''%%/usr/src%%''.
 ====== Adding a new top-level directory to /usr/tests/ ======
@@ Line 44: / Line 44: @@
 <callout type="info">
-If you are looking to add kernel-level tests and don't find a ''%%src/sys/tests%%'' directory, that's expected.  Kernel-level tests are supposed to live in ''%%src/tests/sys/%%'' due to build system oddities.
+If you are looking to add kernel-level tests and don't find a ''%%sys/tests%%'' directory, that's expected.  Kernel-level tests are supposed to live in ''%%tests/sys/%%'' due to build system oddities.
 </callout>
 To do this, follow these steps (which assume you would like to register a ''%%tests%%'' subdirectory for ''%%usr.bin%%''):
-  - Edit ''%%src/etc/mtree/BSD.tests.dist%%'' to register the new top-level directory.
+  - Edit ''%%etc/mtree/BSD.tests.dist%%'' to register the new top-level directory.
-  - Edit ''%%src/usr.bin/Makefile%%'' and:
+  - Edit ''%%usr.bin/Makefile%%'' and:
     - Add ''%%src.opts.mk%%'' (after r265419), ''%%bsd.init.mk%%'' or ''%%bsd.own.mk%%'' at the top of the Makefile if needed.  Make sure that the relevant .mk file is included **//after//** any override variables (LIBDIR, etc), otherwise including the .mk file might break the install path, e.g. [[https://svnweb.freebsd.org/base?view=revision&revision=271241|r271241]].
@@ Line 68: / Line 68: @@
   * [[https://svnweb.freebsd.org/base/head/usr.bin/tr/Makefile?revision=265420&view=markup|usr.bin/tr]]
-  - Create ''%%src/usr.bin/tests/Makefile%%'' and add the lines below to it.  Make sure to replace ''%%usr.bin%%'' with the name of your new directory.  These lines cause the generic ''%%Kyuafile%%'' from ''%%src/tests/Kyuafile%%'' to be installed in your new directory so that tests living in subdirectories can automatically be discovered.
+  - Create ''%%usr.bin/tests/Makefile%%'' and add the lines below to it.  Make sure to replace ''%%usr.bin%%'' with the name of your new directory.  These lines cause the generic ''%%Kyuafile%%'' from ''%%tests/Kyuafile%%'' to be installed in your new directory so that tests living in subdirectories can automatically be discovered.
 <code>
@@ Line 77: / Line 77: @@
 TESTSDIR=       ${TESTSBASE}/usr.bin
-.PATH:          ${.CURDIR:H:H}/tests
+.PATH:          ${SRCTOP}/tests # This is required for tests/Kyuafile
 KYUAFILE=       yes
@@ Line 83: / Line 83: @@
 </code>
-  - Add your tests within ''%%src/usr.bin/tests/%%'' as usual.
+  - Add your tests within ''%%usr.bin/tests/%%'' as usual.
 ====== Adding other intermediate directories ======
@@ Line 91: / Line 91: @@
 As an example, consider the case where we want to install ''%%/usr/tests/usr.bin/foo/bar/baz_test%%''.  If you follow the previous instructions, you would end up with ''%%Kyuafile%%''s in ''%%/usr/tests/usr.bin/%%'' and ''%%/usr/tests/usr.bin/foo/bar/%%'', but not in ''%%/usr/tests/usr.bin/foo/%%''.  To install the missing file:
-  - Edit ''%%src/usr.bin/foo/Makefile%%'' and tell it to recurse into ''%%tests%%'' (same as above).
+  - Edit ''%%usr.bin/foo/Makefile%%'' and tell it to recurse into ''%%tests%%'' (same as above).
-  - Create ''%%src/tests/usr.bin/foo/tests/Makefile%%'' with the same procedure as above.  The only difference is that you will have to tweak the ''%%.PATH:%%'' stanza to add additional ''%%:H%%'' modifiers for every other subdirectory you are traversing and adjust ''%%TESTSDIR%%'' accordingly:
+  - Create ''%%tests/usr.bin/foo/tests/Makefile%%'' with the same procedure as above.  The only difference is that you will have to tweak the ''%%.PATH:%%'' stanza to add additional ''%%:H%%'' modifiers for every other subdirectory you are traversing and adjust ''%%TESTSDIR%%'' accordingly:
 <code>
 # $FreeBSD$
-.include <bsd.own.mk>
+# This isn't required if the test program component is under usr.bin/foo/tests ; see
+# https://svnweb.freebsd.org/base?view=revision&revision=289158
-TESTSDIR=       ${TESTSBASE}/usr.bin/foo
+#TESTSDIR=       ${TESTSBASE}/usr.bin/foo
-.PATH:          ${.CURDIR:H:H:H}/tests
+.PATH:          ${SRCTOP}/tests
 KYUAFILE=       yes
@@ Line 112: / Line 112: @@
 ===== Where can I find sample code? =====
-Refer to ''%%src/share/examples/tests/tests/atf/%%'' for sample test program code and the supporting build code.
+Refer to [[https://cgit.freebsd.org/src/tree/share/examples/tests/tests/atf|share/examples/tests/tests/atf/]] for the sample C/shell test program code and the supporting build code.
+Refer to [[https://cgit.freebsd.org/src/tree/tests/examples|tests/examples/]] for the sample python code.
-This section provides generic documentation on how to write ATF-based test programs both using the C interface and the shell interface.
 ===== Where are the APIs documented? =====
-See ''%%atf-c-api(3)%%'', ''%%atf-c++-api(3)%%'' and ''%%atf-sh-api(3)%%'' for details on the C, C++ and shell APIs respectively.  These manual pages are part of the base system.
+See [[https://www.freebsd.org/cgi/man.cgi?query=atf-c-api&apropos=0&sektion=3&arch=default&format=html|atf-c-api(3)]], [[https://www.freebsd.org/cgi/man.cgi?query=atf-c%2B%2B-api&apropos=0&sektion=3&arch=default&format=html|atf-c++-api]] and [[https://www.freebsd.org/cgi/man.cgi?query=atf-sh-api&apropos=0&sektion=3&arch=default&format=html|atf-sh-api(3)]] for details on the C, C++ and shell APIs respectively.  These manual pages are part of the base system.
+Python interface is not documented yet. Examples of the common use cases can be found in the [[https://cgit.freebsd.org/src/tree/tests/examples|tests/examples/]] directory. ATF pytest interface sources can be found in [[https://cgit.freebsd.org/src/tree/tests/atf_python|tests/atf_python]].
 ===== Do I need to remove any temporary files I create? =====
-No!  Kyua does this automatically for you.  **Every test case** is executed in its process and its own temporary subdirectory.  Having code to remove files or to perform any other kind of cleanup is usually wrong and you should avoid it.  Keep things simple and let the infrastructure do the hard work for you!
+No, you don't if the files/directories are created relative to the starting directory.
+"kyua test" creates a sandbox for test program when the test is executed, so each **test case** is executed as its own process with its own temporary subdirectory. Simply put, removing files relative to the test sandbox isn't required and is discouraged because it adds unnecessary complexity to test programs.
 ===== When do I use ATF_CHECK and when ATF_REQUIRE? =====
@@ Line 129: / Line 132: @@
 You can use this distinction in the following way: use ''%%ATF_REQUIRE%%'' to check the code that "prepares" your test case.  Use ''%%ATF_CHECK%%'' to do the actual functionality tests once all the set up has been performed.
+====== FAQ for GoogleTest test programs ======
+===== Where can I find sample code? =====
+Refer to ''%%contrib/googletest/googletest/samples%%'' for the sample test programs' code; refer to ''%%share/examples/tests/tests/googletest%%'' for the supporting build code.
 ====== FAQ for plain test programs ======
@@ Line 134: / Line 143: @@
 ===== Where can I find sample code? =====
-Refer to ''%%src/share/examples/tests/tests/plain/%%'' for sample test program code and the supporting build code.
+Refer to ''%%share/examples/tests/tests/plain%%'' for sample test program code and the supporting build code.
+====== FAQ for TAP test programs ======
+===== Where can I find sample code? =====
+Refer to ''%%share/examples/tests/tests/tap%%'' for sample test program code and the supporting build code.
+----
+{{tag>HowTo}}