Add writeups/hackintosh

14 files changed, 853 insertions, 0 deletions

diff --git a/writeups/hackintosh/build-qemu_rhel.md b/writeups/hackintosh/build-qemu_rhel.md
new file mode 100644
index 0000000..d30e664
--- /dev/null
+++ b/writeups/hackintosh/build-qemu_rhel.md
@@ -0,0 +1,144 @@
+Since we're going to install stuff in `/usr/local`, the subsystems should be set
+up accordingly. Check if it's already set up on the system. You probably have to
+do both of these if the system is fresh from install.
+
+```sh
+# Check ldconfig
+sudo ldconfig -v | grep /usr/local
+# should yield:
+# /usr/local/lib: (from /etc/ld.so.conf.d/...)
+# /usr/local/lib64: (from /etc/ld.so.conf.d/...)
+
+# Check pkgconf
+echo $PKG_CONFIG_PATH
+# should yield:
+# .../usr/local/lib/pkgconfig:/usr/local/lib64/pkgconfig...
+```
+
+If the ldconfig setting is missing:
+
+```sh
+sudo su -c 'echo "/usr/local/lib" >> /etc/ld.so.conf.d/usr-local.conf'
+sudo su -c 'echo "/usr/local/lib64" >> /etc/ld.so.conf.d/usr-local.conf'
+```
+
+If the pkgconf setting is missing, create the file at
+`/etc/profile.d/usr-local.sh`:
+
+```
+if [ -z "$PKG_CONFIG_PATH" ]; then
+	export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/local/lib64/pkgconfig
+else
+	export PKG_CONFIG_PATH="$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig:/usr/local/lib64/pkgconfig"
+fi
+```
+
+Install the dependencies and tools needed to build qemu.
+
+```sh
+sudo dnf install\
+	git\
+	curl\
+	make\
+	tar\
+	bison\
+	flex\
+	meson\
+	gcc-c++\
+\
+	zlib-devel\
+\
+	openssl-devel\
+\
+	ncurses-devel\
+	bzip2-devel\
+	glib2-devel\
+\
+	usbredir-devel\
+	libusb-devel\
+	pixman-devel\
+	libgudev-devel\
+	spice-protocol\
+	opus-devel\
+	libjpeg-turbo-devel\
+	libepoxy-devel\
+\
+	edk2-ovmf
+```
+
+For when things go wrong (you don't need these unless you're an expert)
+
+```sh
+sudo dnf install gdb setroubleshoot-server
+```
+
+Add fcontext entries for the custom built qemu. This doc may be outdated and the
+fcontext entries for qemu in your distro may have been changed. Check the output
+of the following command to see if there's any discrepancy. Basically, what has
+to be done is making `/usr/local` variants for each qemu executable entry.
+
+```sh
+sudo semanage fcontext -l | egrep '/usr/.*/qemu'
+```
+
+For Rocky 9, the entries looked like this. Feel free to use it if you see no
+difference.
+
+```sh
+sudo semanage fcontext -a -t qemu_exec_t                    '/usr/local/bin/qemu'
+sudo semanage fcontext -a -t virt_qemu_ga_exec_t            '/usr/local/bin/qemu-ga'
+sudo semanage fcontext -a -t qemu_exec_t                    '/usr/local/bin/qemu-kvm'
+sudo semanage fcontext -a -t virtd_exec_t                   '/usr/local/bin/qemu-pr-helper'
+sudo semanage fcontext -a -t virtd_exec_t                   '/usr/local/bin/qemu-storage-daemon'
+sudo semanage fcontext -a -t qemu_exec_t                    '/usr/local/bin/qemu-system-.*'
+sudo semanage fcontext -a -t virt_bridgehelper_exec_t       '/usr/local/libexec/qemu-bridge-helper'
+sudo semanage fcontext -a -t virt_qemu_ga_exec_t            '/usr/local/libexec/qemu-ga(/.*)?'
+sudo semanage fcontext -a -t virt_qemu_ga_unconfined_exec_t '/usr/local/libexec/qemu-ga/fsfreeze-hook.d(/.*)?'
+sudo semanage fcontext -a -t virtd_exec_t                   '/usr/local/libexec/qemu-pr-helper'
+sudo semanage fcontext -a -t qemu_exec_t                    '/usr/local/libexec/qemu.*'
+```
+
+Get the sources. dmg2img is not on RHEL repos either so it has to be built from
+the source as well.
+
+```sh
+git clone https://github.com/Lekensteyn/dmg2img
+curl -LO https://download.qemu.org/qemu-8.2.2.tar.xz
+curl -LO https://www.spice-space.org/download/releases/spice-server/spice-0.15.2.tar.bz2
+```
+
+Build and install.
+
+```sh
+pushd dmg2img
+make && sudo make install
+popd
+
+tar xf spice-0.15.2.tar.bz2
+pushd spice-0.15.2
+./configure
+make -j$(nproc) && sudo make install
+popd
+
+tar xf qemu-8.2.2.tar.xz
+pushd qemu-8.2.2
+./configure \
+	--target-list=x86_64-softmmu \
+	--enable-debug \
+	--enable-docs \
+	--enable-vnc \
+	--enable-spice-protocol \
+	--enable-curses \
+	--enable-libusb \
+	--enable-usb-redir \
+	--enable-libudev \
+	--enable-slirp \
+	--enable-spice \
+	--enable-opengl
+make -j$(nproc) && make -j$(nproc) test && sudo make install
+popd
+
+# qemu-kvm is just a symbolic link to the host arch qemu
+# qemu makefile doesn't do it for us
+sudo ln -s /bin/qemu-system-x86_64 /bin/qemu-kvm
+```
diff --git a/writeups/hackintosh/cloud+libvirt.ko.md b/writeups/hackintosh/cloud+libvirt.ko.md
new file mode 100644
index 0000000..1df3fc6
--- /dev/null
+++ b/writeups/hackintosh/cloud+libvirt.ko.md
@@ -0,0 +1,25 @@
+# QEMU와 Libvirt를 사용하여 클라우드에 해킨토시 돌리기
+([cloud+libvirt.md](cloud+libvirt.md) 한국어 개요)
+
+모바일 앱 개발을 위해 IOS 지원을 해야하는 상황인데, 평소에 내가 맥 유저도 아니고
+맥을 사고서는 프로젝트가 끝나고 잘 안쓰게 될 것이 뻔해서 해킨토시를 클라우드에
+돌리는 방법을 연구하였다. 클라우드에 해킨토시를 올려 사용하므로써, 맥북을 따로
+가지고 다녀야 하지 않는다는 점과 여러 사람이 필요할 때마다 접속하여 사용할 수
+있다는 이점이 있다.
+
+중첩 가상화를 지원하면서 스폿 인스턴스를 손쉽게 사용할 수 있는 클라우드 플렛폼은
+Azure라고 판단하여 Azure에 올려보았다.
+
+Libvirt로 L2 게스트(L0: 클라우드 호스트, L1: 클라우드 VM, L2: 클라우드 VM의
+게스트)를 관리하기 편하며 virt-manager의 간편함과 USB 리디렉션 기능,
+virt-viewer의 윈도 지원 등을 이유로 Libvirt를 사용하여 해킨토시를 구동하기 위해
+발생하는 단점보다 장점이 더 크다고 판단하였다.
+
+작업 부하에 맞게 VM 성능을 자유롭게 늘리고 낮출 수 있는 장점이 있다.
+
+USB 관련 문제가 많다. USB 리디렉션으로 클라우드에서 동작하는 원격의 VM에 로컬
+USB 장치를 제한적으로 연결할 수 있다. 안타깝게도 안드로이드와 IOS 폰의 USB
+리디렉션은 게스트가 로컬에서 동작하면 문제가 없으나, 네트워크를 거치면
+불가능하다. 따라서 IOS 개발은 Ad-hoc 배포를 통해서만 가능하다.
+
+오디오 지원은 아직 펌웨어 수정이 필요하다.
diff --git a/writeups/hackintosh/cloud+libvirt.md b/writeups/hackintosh/cloud+libvirt.md
new file mode 100644
index 0000000..9bb2a05
--- /dev/null
+++ b/writeups/hackintosh/cloud+libvirt.md
@@ -0,0 +1,425 @@
+# Running Hackintosh on Cloud using QEMU and Libvirt
+![virt-manager graphical console of hackintosh](image-7.png)
+
+```
+                <- Libvirt Domain Control ->
+                <- SPICE/VNC ->
+                <- USB Redirection ->
+LOCAL MACHINE           (INTERNET)           Cloud VM
+ virt-manager                                 libvirtd
+ virt-viewer                                   qemu-kvm
+                                                hackintosh
+```
+
+I've found myself in the mobile dev scene. I thought it was time to buy macbook
+but I was reluctant to make such a financial commitment for something I'll
+rarely use again after I'm done with the project. Hackintosh came up as a viable
+option so I took a deep dive into the rabbit hole.
+
+My choice of tools are Libvirt and qemu-kvm. I chose them because they're the
+ones I use on the daily basis. The fronts like virt-manager and virsh are quite
+nice, too. I've also decided to try running on a cloud VM so my mates can use it
+when they need it.
+
+My goal is to build and test React Native apps for IOS devices. So my mac
+environment has to have all the tools for it: Xcode, pod, brew and all the other
+shebang. Therefore I had to get **Sonoma** working because the latest version of
+Xcode dropped support for older macos.
+
+Apple Silicon is not discussed here because, as of the time of writing, no
+virtualisation software supports Apple's proprietary ARM instructions. Apple's
+recent push to move over to ARM is a real threat to Hackintosh community. Apple
+will eventually drop support for x86 macs and when that time comes, this post
+will no longer be relevant. Reverse engineering Rosetta is definitely a
+challenge. The law suits that come after will be, too.
+
+To those who are new to Hackintosh scene, emulating Mac has always been a
+constant battle between Apple constantly breaking backward compatibility and the
+community trying to fix broken stuff caused by that. The documents get outdated
+pretty quick so if this post looks old or doesn't look right, I suggest quickly
+moving onto new ones online.
+
+## Acknowledgements
+Many thanks to [kholia](https://github.com/kholia/OSX-KVM), the OpenCore
+project, and the Hackintosh communities.
+
+## Disclaimer
+Building a Hackintosh involves use of arbitrary binary from untrusted sources. I
+took the shortcut by using the binaries built by people on the internet since I
+don't have to do anything "serious" with my Hackintosh, other then IOS app
+building and debugging. At any rate, you may want to build the EFI image,
+OpenCore, and all of its extension, reviewing all the code in the process...
+well,
+
+I recommend buying the real hardware rather than hacking your way to save some
+money by using Hackintosh if you need Mac to do serious business. That way,
+there's no funny business between you and Apple, and you will have someone to
+sue when things go wrong - you're also paying for liability along with warranty
+that comes with the products.
+
+Another option is: "if you can't beat 'em, **DON'T** join 'em". This is the
+option many devs choose to take. If you don't condone Apple's business
+practices, show them by not having anything to do with them. Don't support IOS
+and Mac. Don't buy their overpriced hardware. That's how you vote them out of
+the competition.
+
+## Requirements
+When choosing the instance type for your hackintosh, note that at least 8GB of
+RAM is required[^8] if you want to do anything at all with it. For CPU
+configuration, it depends on the type of CPU the instance has.
+
+### Choice of CSP
+In order for kvm_intel or kvm_amd to work, the host must have virtualization CPU
+extensions enabled(`svm` for AMD and `vmx` for Intel). Since we're looking to
+run qemu-kvm on the cloud VM, the keywords we're looking for are "nested
+virtualization" and, more specifically, "L2" because the guest(L2) is run on a
+cloud VM(L1) which in turn runs on a bare metal hypervisor(L0). For stability
+issues[^1], only a handful of CSPs offer VM types with nested virtualisation
+enabled.
+
+1. Azure[^2]: the most painless and cost-effective option as of 2024. Easy to
+   request spot VMs, bargain spot prices, supported on both AMD and Intel
+1. GCP[^3]: supported on Intel instances only
+1. AWS: nested virtualisation support is officially supported only on bare metal
+   instances[^4]. My experience is that you *can* use KVM on compute optimized
+   instances[^5]. Worth a try if you feel so adventurous. Painful to launch spot
+   VMs because spot VMs can only be purchased via fleet requests on AWS
+
+If you look at the references closely, you'll see that nested virtualisation on
+CSP hypervisors is a cutting-edge technology. Your options are pretty much
+limited to the big 3.
+
+### Azure
+Running the vm on an E series instance was not feasible because the vCores are
+hyper-threaded and credit limited.
+
+**D4as_v5** w/ 14GB of RAM allocated to the vm was alright. **D8as_v5** w/ 28GB
+was perfect for my use case(VSCode, building React Native app, Xcode, and one
+iPhone SE 2nd simulator).
+
+The bandwidth of "Standard SSD" on Azure is in fact not "standard" at all. You
+need to use "Premium SSD" to get AWS's gp2/3 performance or the vm will
+bottleneck upon disk IO. At least 50GB is required. After installing all the
+apps for my set up it took up over 50GB meaning 100GB or more is required. So I
+recommend starting with a **premium SSD 128GB**.
+
+#### TCP Idle Timeout
+Azure has aggressive default TCP idle connection cutout(4 minutes)[^6]. Rather
+than fiddling with the cloud config, it'll be best if you just use even a more
+aggressive keepalive interval on the software you have control over(sshd or
+sysctl) so you don't have to worry about changing it every time you deploy a
+public IP on Azure. Since ssh tunneling is used to control libvirt, uncomment
+and change the values of the following settings in sshd_config.
+
+```
+TCPKeepAlive yes
+ClientAliveInterval 30
+ClientAliveCountMax 3
+```
+
+### Building QEMU(for Apple SMC device emulation)
+Some distros don't ship qemu package with Apple SMC device emulation. This is
+strange because this can only be done by patching qemu rather than the build
+config. Someone at Red Hat went a great length to remove it. But the feature
+is included in Fedora's PRM.
+
+So, you're left with 2 options:
+
+1. Pick a distro(**Fedora**) that ships qemu with Apple SMC
+1. Build qemu yourself
+
+To see if the qemu is built with SMC emulation, run:
+
+```sh
+echo -e 'info qtree\nq\n' |
+	qemu-kvm
+		-machine q35
+		-device 'isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc'
+		-display none
+		-monitor stdio
+```
+
+The qemu should run and exit normally. The output should contain something like:
+
+```
+dev: isa-applesmc, id ""
+  iobase = 768 (0x300)
+  osk = "ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc"
+```
+
+If you end up with errors it means that you have to build qemu yourself. If you
+don't have to build qemu yourself, skip to [Download OSX
+Images](#download-osx-images).
+
+#### RHEL
+For longevity of the VM image, I felt that it is necessary to set it up on
+Rocky. Following is how I built mine for use on my Rocky instance. Remember that
+this is a note, not a full-blown guide on how to Hackintosh so please try to
+read between the lines before contacting me for help.
+
+Continued in [build-qemu_rhel.md](build-qemu_rhel.md).
+
+## Download OSX Images
+On the host,
+
+```sh
+# Clone the repo
+git clone https://github.com/kholia/OSX-KVM
+cd OSX-KVM
+# Run the script to download the installer image
+./fetch-macOS-v2.py
+# (choose Sonoma or higher)
+# Convert the image
+dmg2img BaseSystem.dmg
+```
+
+Bring the images down to the libvirt image pool.
+
+```sh
+sudo cp BaseSystem.img OpenCore/OpenCore.qcow2 /var/lib/libvirt/images
+```
+
+## Setting Up
+Install libvirt and its friends on the instance.
+
+```sh
+sudo dnf install \
+	libvirt \
+	libvirt-daemon \
+	libvirt-daemon-kvm \
+	libvirt-daemon-driver-interface \
+	libvirt-daemon-driver-network \
+	libvirt-daemon-driver-nodedev \
+	libvirt-daemon-driver-nwfilter \
+	libvirt-daemon-driver-secret \
+	libvirt-daemon-driver-qemu
+```
+
+Enable and fire them up.
+
+```sh
+sudo systemctl enable --now \
+	libvirtd.service \
+	virtqemud.service \
+	virtlogd.service
+```
+
+Register your account to the groups.
+
+```sh
+sudo usermod -aG libvirt "$USER"
+```
+
+On your machine, install `virt-manager`.
+
+```sh
+# (RPM)
+sudo dnf install virt-manager
+# (DEB)
+sudo apt-get install virt
+# (Gentoo)-manager
+sudo emerge virt-manager
+# (OpenBSD)
+sudo pkg_add virt-manager
+# (mac)
+brew install virt-manager
+```
+
+Run virt-manager.
+
+### Add Libvirt Host Connection
+Add connection to the host(`File` - `Add Connection`).
+
+![virt-manager add connection dialog](image.png)
+
+Connect to the libvirt daemon running on the host by double clicking on the name
+or using the context menu.
+
+## Generate Unique Values for the VM (optional)
+The hackintosh will function just fine without changing the machine serial
+numbers that come with kholia's OpenCore loader. If you don't want to take any
+chances, go ahead and generate the serial for your hackintosh.
+
+https://github.com/sickcodes/osx-serial-generator
+
+Use the script `generate-unique-machine-values.sh` to a generate machine values.
+You can use the script to build the OpenCore boot disks with the values inserted
+to the images. Follow the repo's instruction to do that. Here, I'll demonstrate
+how to set them up manually.
+
+The script writes the values to csv and tsv files. Access them anytime by
+reading the files.
+
+## Create the hackintosh VM
+Add a VM on the host. I'm laying down 2 ways to do this: create one from
+scratch(DIY) using virt-manager and import the one I made.
+
+### Import the XML
+[mac.libvirt.xml](mac.libvirt.xml) is my latest working copy. The Upload the
+domain xml to the host and import using virsh.
+
+```sh
+sudo virsh define mac.libvirt.xml
+# Copying from your machine and pasting using the stdin also works:
+# sudo virsh define /dev/stdin
+```
+
+After importing the xml, edit the memory and the number of vcpu to your needs.
+
+Optional: manually edit the MAC of the net interface to the one generated
+beforehand.
+
+```xml
+<!-- Paste the unique mac generated for your vm(optional) -->
+<qemu:arg value="virtio-net-pci,addr=0x12.0x0,netdev=net0,id=net0,mac=THE_MAC"/>
+```
+
+### DIY(virt-manager)
+See [libvirt-diy.md](libvirt-diy.md).
+
+## Attach Volumes and Start the VM
+Create a system volume for the vm. **50GB** is a good number to start with.
+**100GB** to be on the safe side. Attach it to the vm as a SATA disk. You don't
+have to register the disk in the boot order because only OpenCore can load the
+OS.
+
+Add another SATA disk with the macOS image(`BaseSystem.img`). Again, doesn't
+have to be in the boot order as the image is loaded by OpenCore. The disk can be
+removed from the vm after the installation.
+
+You may now start the VM.
+
+## Setting Serial Number (optional)
+You can set the machine serial numbers by
+
+1. Using the EFI shell
+1. Mounting the OpenCore disk on the host and editing the config file
+
+In the OpenCore boot image is the file `config.plist`, which contains the serial
+numbers for the machine. The first method is illustrated here.
+
+![UEFI Shell selection](image-2.png)
+
+Drop to UEFI shell.
+
+![UEFI shell capture](image-3.png)
+
+Find `config.plist` and run `edit` on it.
+
+![UEFI edit command](image-4.png)
+
+**F4** key to search string `PlatformInfo`. Type in the values you generated in
+the [previous step](#generate-unique-values-for-the-vm-optional).
+
+**F3** to exit. Save before exit.
+
+![Reset NVRAM selection](image-5.png)
+
+`exit` to return to the OpenCore boot menu. Do a NVRAM reset.
+
+## Done!
+Boot to **BaseSystem** and install the OS. Start using the hackintosh.
+
+## Cool Stuff
+### Scaling up and down
+Unlike Windows, macOS doesn't seem to be bothered by hardware change. The VM can
+be scaled and down without any modification to the parameters or reinstallation.
+
+![Hackintosh running with 64 cores and 192GB of RAM](image-6.png)
+
+Meaning, something like this is possible(tested by myself). And, of course, the
+VM can be scaled down once the heavy lifting is done. This can be done
+painlessly on Azure because, unlike AWS, Azure lets you change the size of a
+spot VM without deleting or recreating the instance.
+
+## Caveats/TODOs
+With all the effort, at the end of the day, nothing beats buying a real mac.
+There are many caveats when it comes to using Hackintosh in general. Even more
+from using Libvirt's abstraction and running it on the cloud.
+
+Unlike Windows, Mac(and IOS for that matter) doesn't have to support a wide
+range of products. This enabled Apple devs to make assumptions about underlying
+hardware and take many shortcuts when maintaining the kernel. The virtualization
+software products in the market are not simply cut out for emulating Mac.
+
+The qemu passthrough part of the XML looks ugly but this is the cost of running
+Hackintosh on Libvirt. In my opinion, Libvirt is a good tool and the benefit of
+using it outweighs the cost.
+
+### Q35 pcie-root-port
+When the machine type is set to Q35, libvirt "forcibly" adds pcie-root-port to
+the root PCI-E controller. This is to implement PCI-E device hotplug[^10] and
+modern kernels have no issue walking through these "root ports". For Mac, the
+closest thing it has to hot plug are Thunderbolt devices and supporting these
+"root ports" doesn't make any sense on Mac. All the PCI-E devices Libvirt
+attaches to the VM are attached on root ports, which are non-existent on Mac
+hardware hence rendering them useless for Hackintosh.
+
+To get around this issue, the XHCI controller and the input devices need to be
+attached directly to the PCI-E root via qemu command line passthrough
+
+Someone pointed out the issues of Libvirt forcing root ports, but the libvirt
+devs said no[^9].
+
+### XHCI Related Issue
+The USB 2.0 controller support has been dropped since Sonoma(14) so the input
+devices(usb-kbd and usb-tablet) need to be attached directly to the XHCI
+controller[^11].
+
+There is a discrimination as to types of device that can be attached to the
+controller and the ones that can only be attached to a hub. The XNU kernel is
+unable to enumerate a USB drive when it is attached to the controller. So all
+the usb-redir devices are attached to the usb-hub, which operates at full-speed.
+
+### No Sound
+AppleALC is included in the boot image, but for reasons unknown it doesn't seem
+to be working in Sonoma(14). This might be a simple matter of adding the qemu
+hda codec to AppleALC. It seems that the qemu's hda never made it to
+AppleALC in the first place[^13].
+
+### USB Redirection
+See [usbredir.md](usbredir.md).
+
+### SMM Instruction Crash Bugs
+QEMU crashes when the secboot variant of stock edk2-ovmf is used. When the Mac
+boots up and seconds later, the SMM[^7] instructions are issued and qemu crashes
+with SIGABRT. This appears to be caused by a bug in the kvm module, not QEMU.
+
+https://gitlab.com/qemu-project/qemu/-/issues/1198
+
+So the SMM and secure boot have to be turned off.
+
+```xml
+<loader readonly="yes" secure="no" type="pflash">...</loader>
+...
+<smm state="off"/>
+```
+
+### Notes on Migration/Teleportation
+The instance cannot be saved to disk because of the `invtsc`.
+
+VMWare calls it teleportation. Others call it migration. It being: pausing and
+saving the state of the vm to continue its execution on another bare metal
+hypervisor.
+
+The CSP could migrate your VM running a guest without notice. In such event,
+your Hackintosh will suffer the side effects as there are parameters that make
+the qemu guest difficult to save in order to migrate.
+
+## Links
+- https://hackintosh.com/
+- https://www.tonymacx86.com/
+
+[^1]: https://linux-kvm.org/page/Nested_Guests#Limitations
+[^2]: https://azure.microsoft.com/en-us/blog/nested-virtualization-in-azure/
+[^3]: https://cloud.google.com/compute/docs/instances/nested-virtualization/overview
+[^4]: https://aws.amazon.com/about-aws/whats-new/2021/02/introducing-amazon-ec2-m5n-m5dn-r5n-and-r5dn-bare-metal-instances/
+[^5]: https://repost.aws/questions/QUkOwmVhagQbOumNhdfc4YcA/
+[^6]: https://carsonip.me/posts/azure-tcp-idle-timeout-tcp-keepalive-python/
+[^7]: https://en.wikipedia.org/wiki/System_Management_Mode
+[^8]: https://en.wikipedia.org/wiki/MacBook_Air_(Intel-based)#Retina_(2018%E2%80%932020)
+[^9]: https://listman.redhat.com/archives/libvir-list/2020-February/198033.html
+[^10]: https://libvirt.org/pci-hotplug.html
+[^11]: https://github.com/kholia/OSX-KVM/pull/238
+[^12]: https://github.com/kholia/OSX-KVM/blob/master/macOS-libvirt-Catalina.xml#L166
+[^13]: https://github.com/acidanthera/AppleALC/wiki/Supported-codecs#currently-supported-codecs-2024-02-05-v189
diff --git a/writeups/hackintosh/image-1.png b/writeups/hackintosh/image-1.png
new file mode 100644
index 0000000..d0044f1
--- /dev/null
+++ b/writeups/hackintosh/image-1.png
diff --git a/writeups/hackintosh/image-2.png b/writeups/hackintosh/image-2.png
new file mode 100644
index 0000000..a815cf2
--- /dev/null
+++ b/writeups/hackintosh/image-2.png
diff --git a/writeups/hackintosh/image-3.png b/writeups/hackintosh/image-3.png
new file mode 100644
index 0000000..f933913
--- /dev/null
+++ b/writeups/hackintosh/image-3.png
diff --git a/writeups/hackintosh/image-4.png b/writeups/hackintosh/image-4.png
new file mode 100644
index 0000000..74c7239
--- /dev/null
+++ b/writeups/hackintosh/image-4.png
diff --git a/writeups/hackintosh/image-5.png b/writeups/hackintosh/image-5.png
new file mode 100644
index 0000000..e96e8cd
--- /dev/null
+++ b/writeups/hackintosh/image-5.png
diff --git a/writeups/hackintosh/image-6.png b/writeups/hackintosh/image-6.png
new file mode 100644
index 0000000..51515b2
--- /dev/null
+++ b/writeups/hackintosh/image-6.png
diff --git a/writeups/hackintosh/image-7.png b/writeups/hackintosh/image-7.png
new file mode 100644
index 0000000..88250fb
--- /dev/null
+++ b/writeups/hackintosh/image-7.png
diff --git a/writeups/hackintosh/image.png b/writeups/hackintosh/image.png
new file mode 100644
index 0000000..a3ca9a6
--- /dev/null
+++ b/writeups/hackintosh/image.png
diff --git a/writeups/hackintosh/libvirt-diy.md b/writeups/hackintosh/libvirt-diy.md
new file mode 100644
index 0000000..48a15d5
--- /dev/null
+++ b/writeups/hackintosh/libvirt-diy.md
@@ -0,0 +1,99 @@
+![Add VM icon](image-1.png)
+
+Things to set/look out for:
+
+- "Manual Install": as OpenCore loader can only deal SATA disks, you'll have to
+  add the install media yourself after the creation process
+- "Generic or unknown OS"
+- Memory: leave about 2GB to the host
+- CPUs: match or fewer than host's count
+- Disk image size: create one during the process or manually after creation
+- Tick "Customize configuration before install", Finish
+  - Click on "Apply" before leaving the view
+  - Chipset: Q35
+  - Firmware: UEFI
+  - Delete the NIC, audio, tablet, usb redirection
+  - Change the IDE disk to SATA
+  - Manually edit the xml
+    - Change the EFI images to the non secboot variant
+    - Turn smm off
+
+```xml
+<loader readonly="yes" secure="no" type="pflash">/usr/share/edk2/ovmf/OVMF_CODE.fd</loader>
+<nvram template="/usr/share/OVMF/OVMF_VARS.fd"></nvram>
+...
+<smm state="off"/>
+```
+
+Start editing the XML manually. Here's the breakdown of the changes.
+
+Add the xmlns spec. Without this, `<qemu:commandline>` tag won't persist.
+Without the tag, the xmlns spec won't persist either, so add the command line
+passthrough first.
+
+The qemu commandline passthrough is unavoidable because it is not possible to
+set up Apple SMC and custom CPU flags in Libvirt. It is also to circumvent the
+issue of libvirt's mandatory use of pcie-root-ports for the Q35 machine type.
+See [cloud+libvirt.md#caveats](cloud+libvirt.md#caveats) for detail.
+
+The PCI addresses are crafted so that they won't conflict with libvirt's
+allocation.
+
+```xml
+<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
+  ...
+  <qemu:commandline>
+    <qemu:arg value="-cpu"/>
+    <!-- "Intel Cascade Lake-based Xeon W" of Mac Pro 2019 -->
+    <qemu:arg value="Cascadelake-Server-noTSX,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc"/>
+    <qemu:arg value="-smbios"/>
+    <qemu:arg value="type=2"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="qemu-xhci,id=xhci,addr=0x10.0x0"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-kbd,bus=xhci.0"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-tablet,bus=xhci.0"/>
+    <!--
+        Attach the usbredir devices to a hub connected to a separate XHCI
+        controller so that the kernel can enumerate them. Connecting them
+        directly to the controller will result in kernel complaining in loop.
+    -->
+    <qemu:arg value="-device"/>
+    <qemu:arg value="qemu-xhci,id=usbr,addr=0x11.0x0"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-hub,bus=usbr.0,port=1"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-redir,bus=usbr.0,port=1.1,chardev=usbredirchardev1,id=usbredirdev1"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-redir,bus=usbr.0,port=1.2,chardev=usbredirchardev2,id=usbredirdev2"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-redir,bus=usbr.0,port=1.3,chardev=usbredirchardev3,id=usbredirdev3"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-redir,bus=usbr.0,port=1.4,chardev=usbredirchardev4,id=usbredirdev4"/>
+    <qemu:arg value="-device"/>
+    <!-- Paste the unique mac generated for your vm(optional) -->
+    <qemu:arg value="virtio-net-pci,addr=0x12.0x0,netdev=net0,id=net0,mac=44:E6:6E:4F:87:9C"/>
+    <qemu:arg value="-netdev"/>
+    <qemu:arg value="user,id=net0"/>
+    <qemu:arg value="-chardev"/>
+    <qemu:arg value="spicevmc,name=usbredir,id=usbredirchardev1"/>
+    <qemu:arg value="-chardev"/>
+    <qemu:arg value="spicevmc,name=usbredir,id=usbredirchardev2"/>
+    <qemu:arg value="-chardev"/>
+    <qemu:arg value="spicevmc,name=usbredir,id=usbredirchardev3"/>
+    <qemu:arg value="-chardev"/>
+    <qemu:arg value="spicevmc,name=usbredir,id=usbredirchardev4"/>
+  </qemu:commandline>
+</domain>
+```
+
+Turn off memballoon.
+
+```xml
+<memballoon model="none"/>
+```
+
+That's pretty much it!
diff --git a/writeups/hackintosh/mac.libvirt.xml b/writeups/hackintosh/mac.libvirt.xml
new file mode 100644
index 0000000..f9198a5
--- /dev/null
+++ b/writeups/hackintosh/mac.libvirt.xml
@@ -0,0 +1,134 @@
+<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">
+  <name>mac-a</name>
+  <uuid></uuid>
+  <memory unit="KiB">8388608</memory>
+  <currentMemory unit="KiB">8388608</currentMemory>
+  <vcpu placement="static">4</vcpu>
+  <os>
+    <type arch="x86_64" machine="q35">hvm</type>
+    <loader readonly="yes" secure="no" type="pflash">/usr/share/edk2/ovmf/OVMF_CODE.fd</loader>
+    <nvram template="/usr/share/OVMF/OVMF_VARS.fd"></nvram>
+  </os>
+  <features>
+    <acpi/>
+    <apic/>
+    <vmport state="off"/>
+    <smm state="off"/>
+  </features>
+  <cpu mode="host-passthrough" check="none" migratable="on"/>
+  <clock offset="utc">
+    <timer name="rtc" tickpolicy="catchup"/>
+    <timer name="pit" tickpolicy="delay"/>
+    <timer name="hpet" present="no"/>
+  </clock>
+  <on_poweroff>destroy</on_poweroff>
+  <on_reboot>restart</on_reboot>
+  <on_crash>destroy</on_crash>
+  <pm>
+    <suspend-to-mem enabled="no"/>
+    <suspend-to-disk enabled="no"/>
+  </pm>
+  <devices>
+    <disk type="file" device="disk">
+      <driver name="qemu" type="qcow2"/>
+      <source file="/var/lib/libvirt/images/OpenCore.qcow2"/>
+      <target dev="sdb" bus="sata"/>
+      <boot order="1"/>
+      <address type="drive" controller="0" bus="0" target="0" unit="1"/>
+    </disk>
+    <controller type="sata" index="0">
+      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
+    </controller>
+    <controller type="pci" index="0" model="pcie-root"/>
+    <controller type="pci" index="1" model="pcie-root-port">
+      <model name="pcie-root-port"/>
+      <target chassis="1" port="0x10"/>
+      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
+    </controller>
+    <controller type="pci" index="2" model="pcie-root-port">
+      <model name="pcie-root-port"/>
+      <target chassis="2" port="0x11"/>
+      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
+    </controller>
+    <controller type="pci" index="3" model="pcie-root-port">
+      <model name="pcie-root-port"/>
+      <target chassis="3" port="0x12"/>
+      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
+    </controller>
+    <controller type="usb" index="0" model="qemu-xhci">
+      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
+    </controller>
+    <controller type="virtio-serial" index="0">
+      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
+    </controller>
+    <serial type="pty">
+      <target type="isa-serial" port="0">
+        <model name="isa-serial"/>
+      </target>
+    </serial>
+    <console type="pty">
+      <target type="serial" port="0"/>
+    </console>
+    <channel type="spicevmc">
+      <target type="virtio" name="com.redhat.spice.0"/>
+      <address type="virtio-serial" controller="0" bus="0" port="1"/>
+    </channel>
+    <input type="mouse" bus="ps2"/>
+    <input type="keyboard" bus="ps2"/>
+    <graphics type="spice" autoport="yes">
+      <listen type="address"/>
+    </graphics>
+    <audio id="1" type="spice"/>
+    <video>
+      <model type="qxl" ram="65536" vram="65536" vgamem="16384" heads="1" primary="yes"/>
+      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
+    </video>
+    <watchdog model="itco" action="reset"/>
+    <memballoon model="none"/>
+  </devices>
+  <qemu:commandline>
+    <qemu:arg value="-cpu"/>
+    <!-- "Intel Cascade Lake-based Xeon W" of Mac Pro 2019 -->
+    <qemu:arg value="Cascadelake-Server-noTSX,kvm=on,vendor=GenuineIntel,+invtsc,vmware-cpuid-freq=on,+ssse3,+sse4.2,+popcnt,+avx,+aes,+xsave,+xsaveopt,check"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="isa-applesmc,osk=ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc"/>
+    <qemu:arg value="-smbios"/>
+    <qemu:arg value="type=2"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="qemu-xhci,id=xhci,addr=0x10.0x0"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-kbd,bus=xhci.0"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-tablet,bus=xhci.0"/>
+    <!--
+      Attach the usbredir devices to a hub connected to a separate XHCI
+      controller so that the kernel can enumerate them. Connecting them directly
+      to the controller will result in kernel complaining in loop.
+    -->
+    <qemu:arg value="-device"/>
+    <qemu:arg value="qemu-xhci,id=usbr,addr=0x11.0x0"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-hub,bus=usbr.0,port=1"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-redir,bus=usbr.0,port=1.1,chardev=usbredirchardev1,id=usbredirdev1"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-redir,bus=usbr.0,port=1.2,chardev=usbredirchardev2,id=usbredirdev2"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-redir,bus=usbr.0,port=1.3,chardev=usbredirchardev3,id=usbredirdev3"/>
+    <qemu:arg value="-device"/>
+    <qemu:arg value="usb-redir,bus=usbr.0,port=1.4,chardev=usbredirchardev4,id=usbredirdev4"/>
+    <qemu:arg value="-device"/>
+    <!-- Paste the unique mac generated for your vm(optional) -->
+    <qemu:arg value="virtio-net-pci,addr=0x12.0x0,netdev=net0,id=net0,mac=44:E6:6E:4F:87:9C"/>
+    <qemu:arg value="-netdev"/>
+    <qemu:arg value="user,id=net0"/>
+    <qemu:arg value="-chardev"/>
+    <qemu:arg value="spicevmc,name=usbredir,id=usbredirchardev1"/>
+    <qemu:arg value="-chardev"/>
+    <qemu:arg value="spicevmc,name=usbredir,id=usbredirchardev2"/>
+    <qemu:arg value="-chardev"/>
+    <qemu:arg value="spicevmc,name=usbredir,id=usbredirchardev3"/>
+    <qemu:arg value="-chardev"/>
+    <qemu:arg value="spicevmc,name=usbredir,id=usbredirchardev4"/>
+  </qemu:commandline>
+</domain>
diff --git a/writeups/hackintosh/usbredir.md b/writeups/hackintosh/usbredir.md
new file mode 100644
index 0000000..7b1c744
--- /dev/null
+++ b/writeups/hackintosh/usbredir.md
@@ -0,0 +1,26 @@
+That 10 ms delay is apparently detrimental. It makes sense because USB is meant
+to be work over short distances. The protocol and even the electrical
+characteristics are built upon this assumption. USB frames are short in length
+and polled frequently, and almost no buffering is done in the background as it
+works closely on the hardware. Therefore, performance hit is inevitable and
+instability issues are expected when USB frames are passed over IP.
+
+The usb-redir devices are attached via qemu command line passthrough.
+virt-manager is not "aware" of this and it doesn't even try to see if there usb
+redirection is available.
+
+If you really have to use it , use
+[virt-viewer](https://virt-manager.org/download). It is available on most
+distros and even on Windows.
+
+### List of Devices
+Some usb devices don't like getting passed around over the network.
+
+| DEVICE | LOCAL | OVER IP |
+|-|-|-|
+| Mobile (Android, Apple) | 🫳kinda(unstable) | ❌NO |
+| Flash Drives | ✅YES | 🫳kinda(slow) |
+| HID(keyboard and mouse) | ✅YES | ✅YES |
+
+### Analysis: Android and IOS Devices
+TODO