From: Miod Vallat Subject: Re: where to start with new arch To: S V Cc: tech Date: Tue, 3 Sep 2024 15:23:14 +0000 I wrote the following document in 2008 as a guide for other developers. It's quite dated as there was no llvm in the base system and there are probably quite a few things which need updating, but this should be a good starting point. Miod IMPORTING A NEW PORT INTO THE TREE ================================== The following tries to list the good practice to create a new port, and attemps to list all the files and directories which will need to be created or modified to welcome a new OpenBSD architecture. Comments, questions -> miod@ TOOLCHAIN ========= GCC and binutils should configure themselves properly based on the `machine` and `arch -s` values. - Makefile.cross You will need to add a TARGET_ARCH stanza for cross-compilers to build properly. - share/mk/bsd.own.mk You will need to update ELF_TOOLCHAIN and USE_GCC3 computations. It might also be necessary to alter NOPIC and PICFLAG depending on your architecture capabilities. And cheat a bit by defining NOPROFILE until your profiling code is ready. Of course, you will not be able to build cross compilers until you populate the include files directory for your platform with the minimum MI includes. - sys/arch/PORT/include - sys/arch/PORT_ARCH/include (if compound architecture) Add the MI include files, necessary for cross tools to compile: _types.h Just copy it from an existing platform. You will need to fix the definition of label_t when implementing setjmp/longjmp in the kernel (used by ddb). cdefs.h Just copy it from a platform with the same binary format (a.out or ELF) as the new one. endian.h Just copy it from a platform with the same endianness as the new one. Define __STRICT_ALIGNMENT if it requires strict alignment (which is very likely, unless it's a new breed of vax or x86). exec.h Define the native binary type (NATIVE_EXEC_...), the ELF size and MID if needed, and _NLIST_DO_xxx and _KERN_DO_xxx matching your binary format. If this is an a.out format (why oh why), define the relocation structure. float.h Just copy it from an IEEE754 platform, or fill it with the applicable values. ieee.h Copy it from an IEEE754 platform, and adjust to the real hardware layout of floating point values on your platform. Pay attention to endianness, remember that bitfields can not be larger than int (so you need to split fields larger than sizeof(int) * NBBY bits in chunks), and that the order of bitfields is endianness dependent. Look at arm for an example of this. ieeefp.h (unless floating-point is not IEE754 style) Define fp_except as the same size of the floating point status register, and define the individual exception as the pending bits in it, this will help your libc bits. The fp_rnd enum will define the individual bits for the rounding mode, and should preferrably match the hardware. internal_types.h This file only needs to define __machine_has_unsigned_chars on platforms where this happens (gcc defines __CHAR_UNSIGNED__ in this case, see arm and powerpc). If your platform does not do IEE754 arithmetic, you'll need to define specific floating point types limits there as well (see vax). limits.h Just copy it from an existing platform with the same register width (i.e. the same sizeof(long) as yours). param.h Define MACHINE, _MACHINE, MACHINE_ARCH, _MACHINE_ARCH and MID_MACHINE for now. You will need much more to build the kernel, though. ptrace.h You will need to define at least PT_GETREGS and PT_SETREGS. The order is not important, as long as they start from PT_FIRSTMACH. Please keep them contiguous for simplicity. If your port has separate floating-point registers (and thus a struct fpreg in addition to struct reg), you'll need to define PT_GETFPREGS and PT_SETFPREGS. stdarg.h You should get it (or the important parts of it) from gcc. varargs.h You only need this file if your port is still using gcc 2.95. vmparam.h Define the text, data and stack initial and maximal sizes. You will need to define more things for the kernel, such as memory search strategies and virtual address space boundaries. Add C stubs for the following files, which will get replaced with real code later: mutex.h -- needs dummy struct mutex and the macros to be defined pcb.h -- needs dummy struct pcb and md_coredump pmap.h -- needs dummy struct pmap proc.h -- needs dummy struct mdproc reg.h -- needs dummy struct reg (anh fpreg) setjmp.h -- needs _JBLEN defined signal.h -- needs sig_atomic_t and struct sigcontext defined Add the following empty files, which will get filled later: atomic.h At this point, you should be able to build the cross-binutils and cross-gcc targets (the latter depends on and builds the former). It's time to try and get a kernel running. KERNEL ====== You'll need to populate the arch/PORT a bit more: - sys/arch/PORT/ create and populate with: compile/.cvsignore (contains the names of kernels to build) conf/files.PORT conf/Makefile.PORT conf/GENERIC conf/RAMDISK dev/... (or wherever you put md drivers) PORT/... - sys/arch/PORT_ARCH/ if this is a compound arch, you need a few files there too: conf/files.PORT_ARCH dev/... (if needed) PORT_ARCH/... Your PORT/include and PORT_ARCH/files need to grow. As a minimum, you will need to add the following files: asm.h cpu.h db_machdep.h (unless you don't want ddb, you fool) frame.h intr.h On an SMP-capable architecture, you'll also need: lock.h mplock.h and real mutex.h routines. An md version of rw_cas() is also mandatory. There are a few other files which are not so unnecessary after all: bus.h (unless your port doesn't bus_space or bus_dma) disklabel.h (unless your port doesn't run on disks) Your port will probably not be considered complete without: kcore.h (needed for libkvm and crash dumps) And finally, it is good practice, although not mandatory (but the Knights Who Say Ni, err, the kernel janitors, will frown on you if you don't do this): autoconf.h (all definitions related to kernel autoconf) conf.h (cdev_xxx_init and [bc]dev_decl lines for conf.c) After churning long enough, your kernel will build. You might want to start working on boot blocks now, unless your PORT firmware is so cool you don't need anything. So you'll build sys/arch/PORT/stand and fill it with goodies. It is recommended that you use the MI libsa loadfile code to load your kernel binary. In which case you'll need an extra include file: loadfile_machdep.h USERLAND ======== If kernel boots and asks for a root device, your next step is to build a RAMDISK with goodies^Wbinaries, which will also let you confirm interrupts and DMA work, as well as process creation, scheduling, and system calls. So it's time to write some libc bits, to begin with: - lib/csu/PORT_ARCH Makefile crt0.c md_init.h (unless your port is not ELF) - lib/libc/arch/PORT_ARCH there are a few directories to populate with md code for strategic routines. Some of them have a C code equivalent and can be omitted, look around at the other ports for examples. RAMDISK ======= Build the libs (cross-lib), and then you can try to build the miniroot filesystem for a RAMDISK kernel. This means building in distrib/, after making sure you have up-to-date crunchgen and crunchide installed on your host. - etc/etc.PORT populate with disktab, fbtab, MAKEDEV.md, sysctl.conf, ttys Generate MAKEDEV with ``make M=PORT m4'' in etc/ - distrib/PORT populate (basically copy an existing recent architecture, keep the bsd.rd bits, cut through install.md and ``list'' to build only what you need). If RAMDISK boots, console works and you can label a disk, things are looking good. However this does not prove your signal handling is correct yet! MORE USERLAND ============= If the RAMDISK appears to work, you're in need of tarballs to unpack. You will need a few more things to build (or cross-build) the complete userland: - sys/arch/PORT{,_ARCH}/include profile.h (to build profiled libraries) spinlock.h (to build libpthread) - lib/libkvm kvm_PORT_ARCH.c or kvm_PORT.c (if specific requirements) - lib/libpthread/PORT_ARCH _atomic_lock.c uthread_machdep.c uthread_machdep_asm.S - lib/librtheard/PORT_ARCH _atomic_lock.c (same as libpthread) rfork_thread.S - usr.sbin/afs/src/lwp process.PORT_ARCH.S Note that you can stub the libkvm routines (see e.g. old versions of kvm_vax.c), until you have working kernel crash dumps. You may want to stub the libpthread/librthread bits as well, until your port is stable, and tackle them with the help of the regression tests (which you will need to run soon enough anyway). If your port support ELF shared libraries, you will need this for ld.so to build: - sys/arch/PORT/include - sys/arch/PORT_ARCH/include reloc.h - libexec/ld.so/PORT_ARCH/ you'll need to write the MD relocation handling parts, making sure you use mprotect wisely, and don't forget to invalidate or synchronize caches after applying relocations. Makefile.inc archdep.h ldasm.S rtld_machine.c syscall.h If your port needs to provide MD-specific vital routines to userland, you'll put them in libarch (lib/libarch/PORT_ARCH), but please refrain from doing so unless absolutely necessary (and discussed with other developers). INTEGRATING INTO THE BUILD ========================== If you are successful, you should be running multiuser at this point, fixing bugs here and there. Before you can boldly try a ``make build'', you need to plumb your port in the tree: - sys/Makefile add PORT to SUBDIR - share/tmac/mdoc/doc-common add a line for PORT in LOCAL section - etc/Makefile add PORT to MAKEDEVARCHS - etc/mtree/4.4BSD.dist add md manpage directories entries (man#/PORT, cat#/PORT and ps#/PORT) for sections 1, 2, 3, 4, 5, 8 and 9. Entries for section 2 are however only necessary if your PORT_ARCH provides a libarch, and it doesn't, right? - share/man/man4 create the man4.PORT subdirectory with a minimal Makefile, and add it to the SUBDIR list in the man4 Makefile. As a minimum, you want to provide autoconf.4, intro.4 and mem.4. - share/man/man8 create the man8.PORT subdirectory with a minimal Makefile, and add it to the SUBDIR list in the man8 Makefile. As a minimum, you want to provide the generated MAKEDEV.8 and boot_PORT.8. You can regenerate MAKEDEV.8 with ``make M=PORT man'' in etc/). Now you can bake a muild, as we old-timers say. Be sure to log the build to a file, so you can check it carefully afterwards for odd warnings, overriden errors which should have been fatal, or link getting nuts on your macros. INSTALLATION MEDIA ================== Woohoo, it built! Now the finishing touches: - etc/etc.PORT Makefile.inc - distrib/notes/PORT create and populate, or do not create it and wait for miod to write it for you... - distrib/notes/m4.common if this is a compound architecture (PORT != PORT_ARCH), add a line to the MACHINE_ARCH macro. - distrib/notes/packages if PORT_ARCH is new and supported by FreeBSD or NetBSD, add a reference. - distrib/notes/Makefile if you created the above directory, add it to the ``allarchs'' target - distrib/sets/lists create the md.PORT files. Do a best effort to fill them with correct data, they will be fixed later. And now you can ``make release'' in etc/. If it completes, it is time to ``sh checkflist 43'' (or whatever version we are running nowadays) in distrib/sets/lists. Unless you're damn smart and your sets files are correct, checkflist will complain about missing (or non existing) files. Redirect its output to a file, and fix your sets. Repeat until you pass checkflist (or ask for help on icb, deraadt@, pvalchev@ or miod@ can surely help you). Reroll your tarballs (my preferred way to do it is to temporarily remove the release target dependencies in etc/Makefile, and rerun make release: this will rebuild the INSTALL.PORT file and the tarballs, and recompute the checksums). YOU'RE ALMOST DONE! =================== Upload your snapshot to cvs. Tell developers. Send dmesg pr0n to misc@. Rejoice. Have a beer (this is the real reason why non-beer drinkers can not port BSD to new platforms). Write a blurb for undeadly. Write a PORT.html file, link it from plat.html. A MAINTAINER'S LIFE =================== Run the regression tests. Do them after every build, at least for a while. Compare build logs to spot regressions. Get more people involved in this platform support. Build xenocara. Fix bugs. Try and build some ports. Building emacs will stop Theo from asking you ``does it run emacs?'' every week. Fix bugs. Get more people involved in this platform support. Write support for more hardware. Fix bugs. Get more people involved in this platform support. Try as many devices as possible (usb, pci, or eisa and sbus if it's old iron). Fix bugs. Get more people involved in this platform support. Find more bugs. Fix bugs. Repeat.