Differences

This shows you the differences between two versions of the page.

--- summerofcode2014:portingtoandroidemulator [2014/06/16 20:59] – AlexanderTarasikov
+++ summerofcode2014:portingtoandroidemulator [2022/06/10 03:24] (current) – Link PeopleNames and use MailTo macro KubilayKocak
@@ Line 1: / Line 1: @@
 ===== Porting FreeBSD to Android Emulator =====
-  * Student: astarasikov@%%FreeBSD%%.org
+  * Student: <astarasikov@FreeBSD.org>
-  * Mentor: gavin@freebsd.org
+  * Mentor: [[/GavinAtkinson|GavinAtkinson]]
 ===== Project description =====
 Here I will describe the progress of porting %%FreeBSD%% to run on top of Android Emulator.
 Android Emulator runs on a lot of platforms and is easy to set up. It can be therefore used as a unified platform for hacking %%FreeBSD%% on ARM.
@@ Line 11: / Line 12: @@
 It's a fork of Qemu emulator with a new emulated %%SoC%% and a board called Goldfish. A somewhat nicer UI is also included.
-===== Approach to solving the problem =====
-I plan to look into how linux drivers for the Goldfish board are implemented and write the corresponding analogues for the %%FreeBSD%% kernel.
 ===== Deliverables =====
-==== Midterm ====
-| Check if timer is working fine | DONE | converted to 64-bit counter |
+| Timer Driver | DONE | converted to 64-bit counter |
-| Working PIC (Interrupt controller) | DONE | IRQs come to Framebuffer/PDEV |
+| Working PIC (Interrupt controller) | DONE | works fine |
-| 'pdev' driver (dynamic device detection) | WIP | See notes about FDT |
+| 'pdev' driver (dynamic device detection) | WIP | Just prints info to dmesg |
 | Ethernet driver | DONE | It's SMSC91xx, just needed MMIO and IRQ setup |
-| Framebuffer driver | DONE | Implemented, need working rootfs to test SYSCONS |
+| Framebuffer driver | DONE | Implemented, using VT/FBD (NEWCONS) |
-| NAND driver | TODO | See notes |
+| MMC/SD card driver | DONE |
-| Booting Root FS from NAND | TODO | Needs MTD device |
+| Booting Root FS from MMC | DONE |
+| Prebuilt images for sharing | DONE |
+| Events driver (virtual keyboard and touchscreen) | NOPE |
-==== Final ====
+===== Milestones =====
-| Update to HEAD, push to SVN | TODO |
-| Events driver (virtual keyboard and touchscreen) | TODO |
-| MMC/SD card driver | TODO |
-| Prebuilt images for sharing | TODO |
-| Start porting %%FreeBSD%% to a Nexus 5 phone (with Qualcomm MSM8974 CPU) | TODO |
-===== Milestones =====
   * 19 May: Started coding, pushed a prototype of a kernel that boots and outputs to UART
   * 23 June: Mid-term submission
@@ Line 40: / Line 32: @@
   * 18 August: hard deadline
-===== Notes for midterm =====
+===== The Code =====
-PDEV is a virtual device available in the Android Emulator. Writing and reading its MMIO registers allows to get the list of available emulated devices such as framebuffer, NAND, MMC along with their MMIO address ranges and IRQ numbers. I am currenly not sure how one patches [[/OpenFirmware|OpenFirmware]] Tree at runtime, so all the PDEV does is printing the list of devices to dmesg. On the other hand, it is unlikely that the MMIO ranges or IRQs change since they are hardcoded in the Emulator source code.
-It turned out that the Emulator boots from an emulated NAND chip and MMC is only used for user data, so writing NAND driver is more important now if we want to build distributable images which can be run without additional setup.
+The code is based off the 'HEAD' version of %%FreeBSD%% cloned around the start of June. You can check it out via
-Framebuffer driver should work, I have verified that it initializes and the screen turns white because the buffer is set with 0xff at the end of driver initialization. I was unable to test the virtual console (SYSCONS) though - if we put framebuffer on the FDT device buss, it will call "sc_attach_unit" early and this will result in a null pointer dereference inside SYSCONS code since no virtual console is yet installed. If we put it onto "sysbus" instead, the driver will get probed later, probably after rootfs is mounted. Since I do not have a working MTD driver yet, I have not tested it.
+<code>
+svn co https://socsvn.freebsd.org/socsvn/soc2014/astarasikov
+</code>
-Overall, the project turned out to be easier than planned since Android Emulator's Goldfish board is very similiar to QEMU's VERSATILE which is supproted by %%FreeBSD%%, although as I predicted, more time was spent debugging various initialization issues rather than writing code.
+I'm now preparing to merge it upstream. I keep a github branch which I regularly rebase against master and push cleaned-up code
-I believe it would be good to cleanup cruft from such drivers such as SYSCONS (which leads to font output being copy-pasted into every framebuffer driver and many drivers consisting 70% of NO-OP function stubs) and identify other areas where it would be useful to provide a higher level of abstraction for %%SoC%% drivers. Therefore, if I have a booting userland on emulator by the end of first week of July, I plan to port %%FreeBSD%% onto a real phone, a Google Nexus 5. I believe certain areas which have been recently improved in Linux such as clock management, runtime PM, complex display <-> GPU mappings, may need more attention in the %%FreeBSD%% kernel.
+<code>
+git clone https://github.com/astarasikov/freebsd.git
-===== The Code =====
+cd freebsd
-Currently the source code is at my github repo and I have some unpushed code since writing this page. I plan to push it to the %%GSoC%% SVN by the end of the week.
+git checkout android_goldfish_arm_master
-Use the **android_goldfish_arm_10.0.0** branch at the [[https://github.com/astarasikov/freebsd|repository]]
+</code>
 ===== Useful Links =====
@@ Line 61: / Line 54: @@
 ===== Caveats =====
-Android Emulator supports ARMv5 (ARM926E) and ARM1136 cores, but not the coprocessor register instruction for domain control for these [[/SoCs|SoCs]]. Since we want to run on a vanilla emulator, let's just rely on the fact that the emulator also supports emulating ARM Cortex-A8 and only build %%FreeBSD%% for ARMv7
+Android Emulator supports ARMv5 (ARM926E) and ARM1136 cores, but not the coprocessor register instruction for domain control for these %%SoCs%%. Since we want to run on a vanilla emulator, let's just rely on the fact that the emulator also supports emulating ARM Cortex-A8 and only build %%FreeBSD%% for ARMv7
 The latest version of Android Emulator is identical to QEMU in that it loads the kernel binary to 0x10000 (yes, 64K, not 1M) and jumps there. So let's just do what the versatilepb board does - link %%FreeBSD%% to 0x100000 (1M) and set up a trampoline. It should theoretically be possible to save some memory by linking to 0x10000 but I assume the kernel initially maps memory by 1M sections and that will fail unless additional patching is done.
+Most importantly - the emulator binaries from Android SDK contain a bug in MMU that does not occur in Linux but leads to data corruption in %%FreeBSD%% and being unable to boot kernel. To work around, you can use the prebuilt binaries from https://android.googlesource.com/platform/prebuilts/android-emulator (branch "l-preview").
+===== How to compile Android Emulator on FreeBSD =====
+I have ported the Android Emulator source code to compile natively on %%FreeBSD%%. Currently graphics are broken but console works. I think it should be possible to have it fully working with minor patches.
+<code>
+git clone https://github.com/astarasikov/qemu.git
+https://github.com/astarasikov/android_external_gtest
+cd cd android_external_gtest
+git checkout cm-11.0
+cd ../qemu.git
+git checkout l-preview-freebsd
+bash android-build-freebsd.sh
+</code>
 ===== How to compile =====
-Oh well, need to update with a script to build from an external directory
 <code>
@@ Line 81: / Line 90: @@
 ==== How to build a qemu-compatible zImage ====
-Apprarently the same way as Versatile PB. Though, maybe this should be done by an LD script in kernel instead.
+The same way as Versatile PB.
 <code>
@@ Line 99: / Line 108: @@
 ==== How to run ====
-Create an Android Virtual device (install the SDK, run 'android'). Be sure to select an ARMv7A target (basically, installing an SDK for Android 4.X will make ARMv7A the default target). Name your AVD (Android Virtual Device) '%%FreeBSD%%' (though I assume it will work even if you call it 'Windows 8')
+Create an Android Virtual device (install the SDK, run 'android'). Be sure to select an ARMv7A target (basically, installing an SDK for Android 4.X will make ARMv7A the default target). Name your AVD (Android Virtual Device) '%%FreeBSD%%'.
 Launch it from console specifying the avd name and the kernel file name ('flash' in this case)
@@ Line 106: / Line 116: @@
 emulator -shell-serial stdio -show-kernel -shell -avd FreeBSD  -kernel flash -qemu
 </code>
+You can get a prebuilt kernel from
+https://drive.google.com/file/d/0B7wcN-tOkdeRN0lRUDJKa2pWM0U/edit?usp=sharing
+You can use the rootfs from Raspberry Pi to test. For example, use the following SD Card image:
+http://ftp4.us.freebsd.org/pub/FreeBSD/snapshots/arm/armv6/ISO-IMAGES/10.0/FreeBSD-10.0-STABLE-arm-armv6-RPI-B-20140729-r269271.img.bz2
+To test, add the "sd card" device to the android AVD. Then, go to "~/.android/avd/YOUR_DEV_NAME.avd". Replace sdcard.img with the downloaded image. Also, edit the .ini file and write the correct sd card size in megabytes (you can also do it while configuring the AVD).
+==== Debugging Bootup issues ====
+In this section I describe the approach I used for debugging the initialization
+process of the %%FreeBSD%% kernel in Android Emulator (modified QEMU) on ARM.
+===== VM Setup =====
+I was using Mac OS X 10.9.4 as a Host OS, and I was using %%FreeBSD%% 10.0 in
+a [[/VirtualBox|VirtualBox]] VM for cross-compiling the kernel. I have configured the VM
+to bridge the virtual network adapter with my [[/WiFi|WiFi]] connection.
+My workflow was the following:
+I logged into the VM by SSH, launched the compilation, and then pulled the
+compiled kernel and debugging symbols to a directory in a Host OS. Then, I
+launched the Android Emulator in the Host OS.
+In the subsequent sections, the following conventions are used:
+VM IP Address -> 192.168.1.99
+Working directory in the Host OS -> /Users/alexander/Documents/handhelds/bsd/
+VM %%RootFS%% mounted to Host OS -> /Users/alexander/Documents/handhelds/bsd/mnt/
+===== GDB Tricks =====
+I have chosen the following approach to debugging: use QEMU's GDB server
+and a GDB from a cross-compiler toolchain. I confess I was a bit lazy
+and decided not to build GDB in %%FreeBSD%%, but used the Linaro toolchain
+(on both Linux and Mac OS X).
+I also mounted the %%FreeBSD%% root file system from the VM via the sshfs. The primary
+reason to do so was to allow GDB to find sources (note that we need to use
+substitute-path to convert absolute paths to our local mount path). Besided,
+it allows to use a GUI text editor which in the Host OS, which may be considered
+useful by some people.
+I used the following script (which I named fbsd.gdb) for the GDB and I invoked GDB as
+arm-none-eabi-gdb -x fbsd.gdb.
+<code>
+sshfs 192.168.1.99:/ mnt/
+</code>
+<code>
+target remote tcp::1234
+add-symbol-file ./kernel.debug 0xc0100000
+set substitute-path / /Users/alexander/Documents/handhelds/bsd/mnt/
+</code>
+<code>
+scp 192.168.1.99:/home/alexander/handhelds/flash . || exit -1
+scp 192.168.1.99:/usr/obj/arm.arm/root/handhelds/bsd_head/astarasikov/head/sys/GOLDFISH/kernel.debug . || exit -1
+killall -9 emulator || true
+emulator -shell-serial stdio -show-kernel -shell -avd FreeBSD  -kernel flash -qemu -s -S
+</code>
+===== Increasing Boot Verbosity =====
+One problem that I've faced is no matter how many options I enable
+(DDB, KDB, GDB, VERBOSE_SYSINIT), all loaded modules and init functions
+were printed as hex addresses instead of symbol name strings.
+JFYI, I have pushed the relevant changes to https://github.com/astarasikov/freebsd/tree/android_goldfish_arm_master
+Of course, it would be possible to use GDB to symbolize each address,
+but that would be boring. So, I added a couple hacks to the kernel:
+In kern/kern_module.c
+at function module_register_init, I have added a debugging print to
+echo the module name.
+<code>
+printf(" module name='%s' ", data->name);
+</code>
+In kern/init_main.c, I have replaced the DB_STGY_PROC with DB_STGY_ANY
+to make the kernel try to resolve the function name within all symbols.
+In case the symbol is not resolved, I used the function db_printsym.
+Unlike symbol_name (which used db_search_symbol), this function
+performs a fuzzy lookup, so if the function pointer is not exactly at its
+entry point, it will still print the function name and the offset unlike
+db_search_symbol which will fail and result in printing the HEX address as is.
+<code>
+--- kern/init_main.c	(revision 270039)
++++ kern/init_main.c	(working copy)
+@@ -263,14 +263,24 @@
+ 			const char *func, *data;
+ 			func = symbol_name((vm_offset_t)(*sipp)->func,
+-			    DB_STGY_PROC);
++			    DB_STGY_ANY);
+ 			data = symbol_name((vm_offset_t)(*sipp)->udata,
+ 			    DB_STGY_ANY);
++
+ 			if (func != NULL && data != NULL)
+ 				printf("   %s(&%s)... ", func, data);
+ 			else if (func != NULL)
+ 				printf("   %s(%p)... ", func, (*sipp)->udata);
+ 			else
++			{
++			    printf("   ");
++				db_printsym((vm_offset_t)(*sipp)->func, DB_STGY_ANY);
++				printf("(&");
++				db_printsym((vm_offset_t)(*sipp)->udata, DB_STGY_ANY);
++				printf(")... ");
++			}
++
++			if (0)
+ #endif
+ 				printf("   %p(%p)... ", (*sipp)->func,
+ 				    (*sipp)->udata);
+</code>
+So, hex address like 0xc010a27c(0) becomes something like "some_cool_function+0x20"
+----
+{{tag>Historical}}