Differences

This shows you the differences between two versions of the page.

--- andriygapon:avgsyscons [2011/12/17 19:48] – wip AndriyGapon
+++ andriygapon:avgsyscons [2016/07/21 11:02] (current) – Move user articles under user heirarchy KubilayKocak
@@ Line 18: / Line 18: @@
 So the above cases would have to be properly converted if/when the ''%%Giant%%'' is replaced with the syscons lock.
-The ''%%Giant%%'' is acquired explicitly only in ''%%sckbdevent%%'', ''%%sckbdevent%%'' is always called as ''%%kb_callback.kc_func%%'' and it seems that in most if not all cases it is invoked either from a callout or from an interrupt handler (called via ''%%kbdd_intr%%'').  So it is possible that this use of the ''%%Giant%%'' can be replaced with {{page>mtx_assert(MA_OWNED)&nofooter&noeditbtn}}}.  It seems that ''%%kb_callback.kc_func%%'' must be called with the ''%%Giant%%'' held by necessity as the code that calls it also modifies internal state of the corresponding keyboard driver.
+The ''%%Giant%%'' is acquired explicitly only in ''%%sckbdevent%%'', ''%%sckbdevent%%'' is always called as ''%%kb_callback.kc_func%%'' and it seems that in most if not all cases it is invoked either from a callout or from an interrupt handler (called via ''%%kbdd_intr%%'').  So it is possible that this use of the ''%%Giant%%'' can be replaced with ''%%mtx_assert(MA_OWNED)%%''.  It seems that ''%%kb_callback.kc_func%%'' must be called with the ''%%Giant%%'' held by necessity as the code that calls it also modifies internal state of the corresponding keyboard driver. //NB: ''%%adb_kbd%%'' seems to be an exception.//
+In normal context the kernel doesn't interact with ''%%syscons%%'''s input path.  If kernel requires console input then it is either already in one of special contexts or must enter such a context.  Otherwise the input path is connected to userland via the tty layer.  In this mode the keyboard drivers operate in an interrupt driven mode.  When the kernel uses the input path, then it always polls the ''%%syscons%%'' and via it the keyboard drivers.  Internally the drivers may still be interrupt driven, but they must not report any input events upstream and must wait for being polled.  The keyboards are put into the polled mode via a special method.  An important thing to note here is that the kernel doesn't acquire any designated lock while polling the ''%%syscons%%'', so there should not be any concurrent access to the ''%%syscons%%'' internal state as well as the internal state(s) of the keyboard driver(s) during kernel polling.  So, once again, since the kernel doesn't acquire any lock to ensure that, then the ''%%syscons%%'' and/or the keyboard drivers must provide the necessary concurrency protection.  This can be achieved either via locking or by designing the code in such a way that the polled flag prevents the concurrent code from altering the state in parallel with kernel polling.  In the latter case the polling flag itself must still be lock protected.
+Another point is that the kernel never has to compete with the userland for access to the ''%%syscons%%'' input path as will be shown further.  Any concurrency is always between the kernel-side threads and/or interrupts.
+===== Special contexts and locking =====
+==== 1. Pause after each line mode ====
+This is an early boot context.  This mode is not supported by all the keyboard drivers.  Only drivers that support early initialization (before usual discovery of system resources) can support this mode.  As such, the ''%%ukbd%%'' driver doesn't support this mode.  In this mode interrupts are not enabled yet, there is only a single ''%%thread0%%'' running and so the polling works naturally without any complications.
+It seems that at present the ''%%atkbd%%'' driver doesn't properly handle its own "non-early" initialization if this mode is active.
+==== 2. mountroot prompt ====
+At this stage the interrupts are already enabled and there are multiple threads of execution.  There are still no userland processes.  This is the most complex context from the point of view of locking.  The keyboard driver's interrupt handler may get invoked but it must not interfere with kernel polling.
+So, in this context in particular, the keyboard drivers must ensure that either their interrupt handler do not fire or that they do not modify parts of the internal state that can be concurrently accessed by the polling code path.
+Giant is already held in this context - it is acquired in ''%%start_init%%'' before the ''%%vfs_mountroot%%'' call.
+==== 3. KDB/DDB context ====
+In this context the kernel is essentially single-threaded.  All but one CPUs are paused and only one thread is executing.  In this case there is no need for protection from concurrent access.  Additionally the only executing thread must not get blocked on any locks, so it explicitly should bypass any normal locking.  Polling should work out naturally in this case.
+==== 4. Panic context with SCHEDULER_STOPPED ====
+This context is very similar to the above context by its properties.  The difference is all the locks are automatically bypassed, so there is no need for explicit checks.
+==== 5. Late shutdown context ====
+This is a context where the kernel may ask to press a key for some action.  E.g. after ''%%shutdown -h%%'' the kernel asks "to press any key to reboot".  At this stage the interrupts and the scheduler should still be functional.  The ''%%Giant%%'' is held by the shutdown thread - it's taken in ''%%sys_reboot%%''.
+==== Summary ====
+It's important to note that the normal locking is usable in all of the above contexts except the number 3.  In the contexts 1, 2 and 5 it simply just works, in the context 4 the locking is auto-magically skipped.  So any explicit locking in the code that can be executed in the context 3 must have an explicit check for ''%%kdb_active%%''.
+Another important note is that there are three ways to call into the ''%%syscons%%'' code and into the keyboard drivers code:
+  * from the tty layer
+  * from interrupt handlers and callouts
+  * from the kernel in polling mode
+The first two are already protected by the proper ''%%Giant%%'' locking, so there is no need for additional locking in these code paths.  On the other hand the entry points for the third path must be explicitly protected.  That protection must obey the previous rule about ''%%kdb_active%%''.
+Additional observation is that the kernel never really competes with the userland for access to the keyboard(s).  The kernel polls the console for input either when there is no userland yet or there is no userland already or either ''%%panic%%'' or ''%%kdb%%'' are entered and the userland is "suspended".
+Also, it can be seen that at present there is no concurrent access to the console input from the kernel side too.  So the code doesn't expect this and is not equipped to properly handle such a situation yet.
+One more thing to note is that at present the kernel always accesses the console input in a well-defined locking locking context.  Either it is a "single-threaded"/"lock-less" context of the ''%%panic%%''/''%%kdb%%'' or a rather clean context of a specific system boot or shutdown stage.
+===== Interaction between the input and output paths =====
+The current code has one non-trivial interaction between the output path and input path.  ''%%syscons%%'' tries to provide the following behavior: if a Scroll Lock mode is active and kernel prints something to a console, the the Scroll Lock must be deactivated and correspondingly a Scroll Lock LED on a keyboard must be turned off.  As mentioned above, the kernel may produce output from arbitrary contexts, but turning off the LED requires calling into the keyboard drivers which require specific locking and currently use the ''%%Giant%%''.  So a direct call may result in a LOR and potentially a deadlock (provided that all the locking requirements are properly satisfied).  One way to work around this problem is to execute the action asynchronously via a ''%%taskqueue%%''.
+===== ukbd =====
+The problem with ''%%ukbd%%'' is that it has to interact with rather complex hardware compared to AT compatible keyboard controller, besides also unlike the AT keyboard controller the USB controller can be shared with rather non-trivial peripherals and their drivers.  By necessity the underlying USB subsystem must use multiple threads in most of the contexts.  Thus the ''%%ukbd%%'' driver must honor concurrency and locking strategies of the USB subsystem when the USB threads are active.
+Summary.  Context 1 is not applicable to the  ''%%ukbd%%''.  In contexts 3 and 4 the USB threads do not run, so the ''%%ukbd%%'' doesn't have to acquire any locks (in fact in context 3 it must not acquire any locks) and also it must directly go all the way to hardware via direct calls.  In context 2 the ''%%ukbd%%'' must acquire the necessary locks, the ''%%Giant%%'' currently, if it is called via routines related to polling.  It's important to note that in this context the ''%%ukbd%%'' driver works in interrupt driven mode internally.  The USB threads are running and they perform the actual interaction with the hardware.  So the ''%%ukbd%%'' must not access the hardware directly.  Also, if it has to sleep/wait for the USB threads, then it should release the ''%%Giant%%'' while doing so, so that the USB threads can interact with the ''%%ukbd%%'' under ''%%Giant%%'' protection.  ukbd is detached before context 5 is entered, so it is useless there and that context is irrelevant to ukbd.
+Routines for non-polling access should already be called with the proper locks, that fact can be asserted in the code.