Implement Intel SMAP and kernel patching framework
- Student: OliverPinter
- Mentor: GavinAtkinson
Project description
In first phase, I want to implement the Intel SMAP (Supervisor Mode Access Prevention) technology for x86-64 architecture. In second phase, I plan to implement boot/load time kernel and kernel module patching framework.
Approach to solving the problem
Intel SMAP is a hardware extension to support advanced kernel self-protection. The SMAP technology will prevent unintended data access from kernel to userland memory. The technology will appear in Intel Broadwell architecture in 2014Q2/Q3. Currently there is an emulator - namely Qemu with TCG - which supports this technology.
Runtime kernel/kernel module patching is required, otherwise the processor will fail when processing unknown instruction. Newer processors introducing newer instructions which didn't exist on older one. To solve this situation this framework makes the kernel and kernel modules self-modifiable in common way.
Deliverables
phase #1: - Improved security of FreeBSD kernel in future x86-64 processors
phase #2: - generic framework for boot-time/runtime kernel image and kernel modules patching - elliminate hackish “manual” instruction patching: http://svnweb.freebsd.org/base/head/sys/amd64/amd64/cpu_switch.S?r1=238450&r2=238449&pathrev=238450
Milestones
phase #1:
| 1.1 | May 19 - May 25 | update Intel SMAP knowledege | DONE |
| 1.2 | May 26 - June 8 | update relevant FreeBSD kernel knowledge | DONE |
| 1.3 | June 9 - June 15 | implement/refine trap handler and add/refine required code to relevant parts of kernel | DONE |
| 1.4 | June 16 - June 22 | test and fix | DONE |
phase #2:
| 2.1 | June 23 - June 29 | identify the required places to modify in booting process and kernel module loading process | DONE |
| 2.2 | June 30 - July 6 | design the kernel patching framework | DONE |
| 2.3 | July 7 - July 20 | implement the kernel patching framework | DONE |
| 2.4 | July 21 - July 27 | adapt XSAVE and SMAP instructions to new framework | DONE |
| 2.5 | July 27 - August 3 | optimize implementation | DONE |
| 2.6 | August 3 - EoC August 14 | test, test, fix, test | DONE |
Test Plan
phase #1 - SMAP:
| create a VM image | http://svnweb.freebsd.org/socsvn/soc2014/op/tools/create_smap_test_vm.csh?view=log | DONE |
| write vulnerable kernel module and PoC, and test | http://svnweb.freebsd.org/socsvn/soc2014/op/tests/smap-tester/ | DONE |
| test in qemu with SMAP emulation | http://svnweb.freebsd.org/socsvn/soc2014/op/tools/run_smap_test_vm.csh?view=log | DONE |
phase #2 - kernel patching:
| create a VM image | DONE | |
| boot test in qemu | DONE | |
| kernel module test in qemu | DONE | |
| test in qemu with enabled SMAP | DONE | |
| test on real hardware with XSAVE/XSAVEOPT | DONE | |
| stress test | DONE |
The Code
https://svnweb.freebsd.org/socsvn/soc2014/op/
https://github.com/opntr/opBSD/tree/op/gsoc2014/master
https://github.com/opntr/opBSD/tree/op/gsoc2014/smap
Useful links
http://forums.grsecurity.net/viewtopic.php?f=7&t=3046
https://lwn.net/Articles/517475/
http://www.freebsd.org/doc/en_US.ISO8859-1/articles/committers-guide/subversion-primer.html
https://software.intel.com/sites/default/files/managed/68/8b/319433-019.pdf
http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2012/10/24592_APM_v11.pdf
http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2012/10/24593_APM_v21.pdf
http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2008/10/24594_APM_v3.pdf
http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2012/10/26568_APM_v41.pdf
http://amd-dev.wpengine.netdna-cdn.com/wordpress/media/2012/10/26569_APM_v51.pdf
http://tigcc.ticalc.org/doc/gnuasm.html#SEC109
http://llvm.org/docs/doxygen/html/AsmParser_8cpp_source.html
http://clang.llvm.org/compatibility.html
http://www.ibm.com/developerworks/aix/library/au-gdb.html
https://sourceware.org/binutils/docs/ld/Scripts.html#Scripts
1.1
doc
319433-019.pdf : 10.3; table 2-5; page 887; page 894
CPUID
CPUID.(EAX=07H,ECX=0H):EBX.SMAP[bit 20] = 1
instructions
stac:
STAC sets the AC flag bit in EFLAGS/RFLAGS without affecting other bits. Attempt to execute STAC when CPL > 0 will cause #UD. This instruction's operation is the same in non-64-bit modes and 64-bit mode.
clac:
CLAC clears the AC flag bit in EFLAGS/RFLAGS without affecting other bits. Attempt to execute CLAC when CPL > 0 will cause #UD. This instruction's operation is the same in on-64-bit modes and 64-bit mode.
status bit
EFLAGS.AC
enabling
“System software enables SMAP by setting the SMAP flag in control register CR4 (bit 21).”
1.2
| clang SMAP support | YES | https://github.com/freebsd/freebsd/commit/aa45f148926e3461a1fd8b10c990f0a51a908cc9 | dim@ |
| gcc SMAP support | YES | https://github.com/freebsd/freebsd/commit/72b58728c7463667c6eb881e2f8a0c6d98e8c661 | pfg@ |
| ddb SMAP support | YES | https://github.com/freebsd/freebsd/commit/0fa3283d5a6ef0a3095fd7dd71bf4cea8ecd83ee | jhb@ |
trap handler memo
“If CPL < 3, SMAP protections are disabled if EFLAGS.AC = 1. If CPL = 3, SMAP applies to all supervisor-mode data accesses (these are implicit supervisor accesses) regardless of the value of EFLAGS.AC.” - Intel Ref. # 319433-014 9.3.2
BOOL
check_smap_access_violation(FRAME, USERMODE)
{
if SMAP_DISABLED :
RETURN FALSE;
if CPL == 3 OR RFLAGS@FRAME BITAND EFLAGS.AC :
RETURN FALSE;
RETURN TRUE;
}
1.3
kernel parts
notes
- i386 part originally not part of this project
- i386 part currently not tested
- do NOT compile the kernel with CPUTYPE=core-avx2
2.1
| linker script | sys/conf/ldscript.amd64 |
| elf linker | sys/kern/kern_linker.c |
| sys/kern/link_elf.c | |
| sys/sys/linker.h | |
| nop tables | XXX |
| kernel image patching | sys/amd64/amd64/machdep.c |
| files, which containes stac, clac or xsave | sys/amd64/amd64/support.S |
| sys/amd64/include/asmacros.h | |
| sys/amd64/include/cpufunc.h | |
| sys/amd64/ia32/ia32_exception.S | |
| amd64/amd64/cpu_switch.S |
- potential useable kernel API's:
- linker_file_lookup_set(…)
- pmap_enter(…)
- pmap_kenter(…)
- sys/linker_set.h
- CPU rendezvous
2.2
- basic structures
- lf_selfpatch_t
typedef struct lf_selfpatch {
void *patchable; /* address of patchable instractions */
void *patch; /* address of patch */
short feature_selector;
int feature;
char *comment;
} lf_selfpatch_t;
- elf sections
- selfpatch_metainfo
- selfpatch_patch
- assembly instruction size calculation like in https://github.com/freebsd/freebsd/blob/master/sys/amd64/include/asmacros.h#L217
- patching scenarios
- kernel
→ hammer_time
→ lf_selfpatch(linker_kernel_file)
- module
→ modload
→ lf_selfpatch(linker_module_file)
- module → kernel (patching the kernel image, from newly loaded kernel modules, probably one way to apply security patches run-time)
→ onload
→ lf_patch(linker_module_file, linker_destination_file)
→ smp_rendezvous_cpu(NULL, lf_selfpatch(foo), NULL, arg)
2.3
- implemented kernel image patching on boot
- implemented module patching at preload and on load
- tested kernel image patching on boot
- updated build tools and maintaining tools
- see commits in https://github.com/opntr/opBSD/tree/op/gsoc2014/kpatch
tools
| svn-cherry-pick-from-git.csh | http://svnweb.freebsd.org/socsvn/soc2014/op/tools/svn-cherry-pick-from-git.csh?view=log |
| build_kernel_32bit.csh | http://svnweb.freebsd.org/socsvn/soc2014/op/tools/build_kernel_32bit.csh?view=log |
| build_kernel_64bit.csh | http://svnweb.freebsd.org/socsvn/soc2014/op/tools/build_kernel_64bit.csh?view=log |
| create_smap_test_vm.csh | http://svnweb.freebsd.org/socsvn/soc2014/op/tools/create_smap_test_vm.csh?view=log |
| run_smap_test_vm.csh | http://svnweb.freebsd.org/socsvn/soc2014/op/tools/run_smap_test_vm.csh?view=log |