Differences

This shows you the differences between two versions of the page.

--- testingfreebsdstatus [2013/05/19 02:22] – JustinHibbits
+++ testingfreebsdstatus [2020/10/27 01:43] (current) – Fix 4 links. SashaVigole
@@ Line 9: / Line 9: @@
   * ATF 0.16 has been imported into %%FreeBSD%% CURRENT and apart from some minor patches, all of the needed support has been added to %%FreeBSD%% (gcooper, marcel).
   * ATF and Kyua ports have been created (asomers, bdrewery).
   * MFCing ATF/bmake support (bmake is required by ATF) has been requested by developers (bapt, et al) to help regression test components on 9-STABLE; MFCing to 8.x was considered a bad idea, but all involved parties (bapt, bdrewery, davide, gcooper) agreed that having ports of ATF/bmake for 8.x was a good idea.
@@ Line 19: / Line 18: @@
   * Automatic test "discovery"/handling logic is queued for commit.
   * buildworld/installworld integration is queued for commit.
-  * Building a test release package and being able to install it via bsdinstall is underway (gcooper).
+  * Building a test release package and being able to install it via bsdinstall is incomplete, but work has been done on GitHub to make this come true.
+  * Getting the test infrastructure to run on stable/8 and stable/9 is possible, and all of the necessary bits have been pushed back to stable/8 and stable/9.
+====== Test Infrastructure Status ======
+===== ATF =====
+  * Test fixtures (global cleanup/setup, per-testcase setup) need to be added to ATF to achieve feature parity with other test infrastructures (GNU libcheck, Python unittest testcases, Junit, etc).
+  * As JulioMerino has noted on his GoogleCodeIn page for [[http://code.google.com/p/kyua|ATF/Kyua]], ATF was just the starting point, and Kyua is the evolution of ATF. So, with that in mind, the work that's being done here is just to provide some degree of feature parity with %%NetBSD%%, etc, and as time progresses, %%FreeBSD%% will catch up with Kyua (which is more evolved than ATF) and will use the test infrastructure [[http://code.google.com/p/kyua/wiki/ATF#Status_of_the_ATF_project|layering]] that Julio Merino has proposed.
+===== RUMP =====
+[[http://www.netbsd.org/docs/rump/|Rump]] is an extension to the %%NetBSD%% kernel that runs kernel code as a standalone user process for development and testing purposes. It is a great technology for isolating code and testing it using a whitebox method of testing, as opposed to having to use fuzzing techniques, failpoints, blackbox/graybox testing, etc, which ultimately allows developers and testers to write more code that isolates API calls and helps ensure that KPI contracts are sane without panicing a system or corrupting system state.
 ====== Testcase Status ======
@@ Line 26: / Line 37: @@
 | Directory | Repository | Owner | Status | Notes |
+| bin/cat | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. |
 | bin/date | [[#yatrc|1]] | gcooper | complete | Ported from tools/regression. Research adding coverage from %%NetBSD%%. |
+| bin/dd | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. Needs greater coverage/porting. Throws false positives. |
+| bin/expr | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. |
 | bin/sh | [[#yatrc|1]] | gcooper | complete | Ported from tools/regression. Need to get jilles@ to stop committing files to tools/regression :P. |
+| bin/sleep | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. |
 | gnu/usr.bin/diff | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. |
-| gnu/usr.bin/grep | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. |
 | lib/libc | [[#yatrc|1]] | gcooper | in-progress | Ported from %%NetBSD%%. Needs greater coverage/porting. Throws false positives. |
 | lib/libcrypt | [[#yatrc|1]] | gcooper | complete? |  |
@@ Line 35: / Line 49: @@
 | lib/libthr | [[#yatrc|1]] | gcooper | in-progress | Ported from %%NetBSD%%. Needs greater coverage/porting. Throws false positives. |
 | lib/msun | [[#yatrc|1]] | gcooper | in-progress | Ported from %%NetBSD%%. Needs greater coverage/porting. Throws false positives. |
+| sbin/ifconfig | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. Needs greater coverage/porting. Throws false positives. |
 | usr.bin/apply | [[#yatrc|1]] | gcooper | complete | Ported from tools/regression. Research adding coverage from %%NetBSD%%. |
+| usr.bin/basename | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. |
 | usr.bin/calendar | [[#yatrc|1]] | gcooper | complete | Ported from tools/regression. |
 | usr.bin/comm | [[#yatrc|1]] | gcooper | complete | Ported from tools/regression. |
 | usr.bin/cut | [[#yatrc|1]] | gcooper | complete | Ported from tools/regression. |
+| usr.bin/dirname | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. |
 | usr.bin/file2c | [[#yatrc|1]] | gcooper | complete | Ported from tools/regression. |
+| usr.bin/grep | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. |
+| usr.bin/gzip | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. |
 | usr.bin/join | [[#yatrc|1]] | gcooper | complete | Ported from tools/regression. |
 | usr.bin/jot | [[#yatrc|1]] | gcooper | complete | Ported from tools/regression. |
@@ Line 52: / Line 71: @@
 | usr.bin/yacc | [[#yatrc|1]] | gcooper | complete | Ported from tools/regression. |
 | usr.sbin/nmtree | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. Adjusted slightly to match sha256sum string. |
-| tests/contrib/atf | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. All test issues have been pushed back via GoogleCode page for the contrib/atf issues; the build issues and adjustments are all remaining local to %%FreeBSD%% for now.. |
+| usr.sbin/tcpdump | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. Needs greater coverage/porting. Throws false positives. |
+| tests/contrib/atf | [[#yatrc|1]] | gcooper | complete | Ported from %%NetBSD%%. All test issues have been pushed back via GoogleCodeIn page for the contrib/atf issues; the build issues and adjustments are all remaining local to %%FreeBSD%% for now.. |
 ===== Test::Harness Testcases =====
@@ Line 66: / Line 86: @@
 | usr.sbin/newsyslog | [[#yatrc|1]] | gcooper | in-progress | From tools/regression. Convert to ATF. Need to investigate failures. |
 | usr.sbin/sa | [[#yatrc|1]] | gcooper | in-progress | From tools/regression. Convert to ATF. Need to investigate failures. |
-===== Repositories =====
-  -  [[https://github.com/yaneurabeya/freebsd/tree/atf-tools-regression-convert|yaneurabeya/.../atf-tools-regression-convert]]