This is an old revision of the document!
Process to update OpenSSL in base
Preliminary steps
1. Move current version of OpenSSL
$ mv /usr/src/crypto/openssl /usr/src/crypto/openssl-1.0.2
2. Download, verify
$ fetch https://www.openssl.org/source/openssl-1.1.0h.tar.gz openssl-1.1.0h.tar.gz 100% of 5295 kB 28 MBps 00m00s $ fetch https://www.openssl.org/source/openssl-1.1.0h.tar.gz.asc openssl-1.1.0h.tar.gz.asc 100% of 455 B 4490 kBps 00m00s $ sha256 ~/openssl-1.1.0h.tar.gz SHA256 (/root/openssl-1.1.0h.tar.gz) = 5835626cde9e99656585fc7aaa2302a73a7e1340bf8c14fd635a62c66802a517 $ grep 5835626cde9e99656585fc7aaa2302a73a7e1340bf8c14fd635a62c66802a517 /usr/ports/security/openssl-devel/* /usr/ports/security/openssl-devel/distinfo:SHA256 (openssl-1.1.0h.tar.gz) = 5835626cde9e99656585fc7aaa2302a73a7e1340bf8c14fd635a62c66802a517 $ gpg --verify openssl-1.1.0h.tar.gz.asc openssl-1.1.0h.tar.gz gpg: WARNING: using insecure memory! gpg: please see http://www.gnupg.org/documentation/faqs.html for more information gpg: Signature made Tue Mar 27 15:50:44 2018 CEST using RSA key ID 0E604491 gpg: Good signature from "Matt Caswell <matt@openssl.org>" gpg: aka "Matt Caswell <frodo@baggins.org>" Primary key fingerprint: 8657 ABB2 60F0 56B1 E519 0839 D9C4 D26D 0E60 4491
3. Unpack, remove version from directory
$ tar xf ~/openssl-1.1.0h.tar.gz -C /usr/src/crypto $ mv /usr/src/crypto/openssl-1.1.0h /usr/src/crypto/openssl
4. Update /usr/src/secure/lib/libcrypto/Makefile.inc
... # OpenSSL version used for manual page generation OPENSSL_VER= 1.1.0h OPENSSL_DATE= 2018-03-27 ...
5. Required tools: Perl
We need pod2man from perl and perl to generate assembly from the perl scripts
$ pkg install perl5 ...
Generate assembly .S files
From secure/libcrypto/Makefile.asm:
# $FreeBSD: head/secure/lib/libcrypto/Makefile.asm 307976 2016-10-26 20:02:22Z jkim $ # Use this to help generate the asm *.S files after an import. It is not # perfect by any means, but does what is needed. # Do a 'make -f Makefile.asm all' and it will generate *.S. Move them # to the arch subdir, and correct any exposed paths and $ FreeBSD $ tags.
Let's do that!
make -f Makefile.asm all [Creating objdir /usr/obj/usr/src/amd64.amd64/secure/lib/libcrypto...] ( echo '/* $'FreeBSD'$ */' ; echo '/* Do not modify. This file is auto-generated from aes-x86_64.pl. */' ; env CC=cc perl /usr/src/crypto/openssl/crypto/aes/asm/aes-x86_64.pl elf ) > aes-x86_64.S ( echo '/* $'FreeBSD'$ */' ; echo '/* Do not modify. This file is auto-generated from aesni-mb-x86_64.pl. */' ; env CC=cc perl /usr/src/crypto/openssl/crypto/aes/asm/aesni-mb-x86_64.pl elf ) > aesni-mb-x86_64.S ... ( echo '/* $'FreeBSD'$ */' ; echo '/* Do not modify. This file is auto-generated from sha512-x86_64.pl. */' ; cat sha256-x86_64.s ) > sha256-x86_64.S env CC=cc perl /usr/src/crypto/openssl/crypto/sha/asm/sha512-x86_64.pl elf sha512-x86_64.s ( echo '/* $'FreeBSD'$ */' ; echo '/* Do not modify. This file is auto-generated from sha512-x86_64.pl. */' ; cat sha512-x86_64.s ) > sha512-x86_64.S
TODO: Verify that we have all necessary .S files
The files processed are defined in the Makefile, which is thus version-specific.
Manpage generation
TL;DR
After fixing Makefile.inc MKLINK output
$ make -C /usr/src/secure/lib/libcrypto man-makefile-update $ make -C /usr/src/secure/lib/libcrypto man-update $ make -C /usr/src/secure/lib/libssl man-makefile-update $ make -C /usr/src/secure/lib/libssl man-update $ make -C /usr/src/secure/usr.bin/openssl man-makefile-update $ make -C /usr/src/secure/usr.bin/openssl man-update
Work log:
Two targets in Makefile.inc: man-update and man-makefile-update.
man-makefile-update
Makefile.man is included for installing the manpages, the first lines indicate this is generated by the man-makefile-update update. Let's run that!
$ make man-makefile-update
rm -f /usr/src/secure/lib/libcrypto/Makefile.man
echo '# $'FreeBSD'$' >> /usr/src/secure/lib/libcrypto/Makefile.man
echo '# DO NOT EDIT: generated from man-makefile-update target' >> /usr/src/secure/lib/libcrypto/Makefile.man
for i in crypto; do for j in /usr/src/crypto/openssl/doc/$i/*.pod; do test -f $j || continue; fn=`basename $j .pod`; if ! echo 'SSLeay_version des_modes' | grep -qw "$fn"; then echo "MAN+= $fn.3"; fi; done; done | env LANG=C s
ort >> /usr/src/secure/lib/libcrypto/Makefile.man
for i in crypto; do for j in /usr/src/crypto/openssl/doc/$i/*.pod; do test -f $j || continue; fn=`basename $j .pod`; if ! echo 'SSLeay_version des_modes' | grep -qw "$fn"; then perl /usr/src/crypto/openssl/util/extract-names.pl <
$j | awk "/^$fn\$/ { next; } { print \"MLINKS+= $fn.3 \" \$1 \".3\" }"; fi; done; done | env LANG=C sort >> /usr/src/secure/lib/libcrypto/Makefile.man
Can't open perl script "/usr/src/crypto/openssl/util/extract-names.pl": No such file or directory
(repeat many times)
Big OOPS! Something changed in upstream OpenSSL that breaks this process. extract-names.pl no longer exists, replaced by process_docs.pl.
Looks like first loop succeeds (MAN+=), but second loop (extract manpage aliases/links) does not (MLINKS+=).
man-update
Let's try man-update target
$ make man-update ASN1_INTEGER_get_int64.3 ASN1_OBJECT_new.3 ... o2i_SCT_LIST.3 x509.3
Converts .pod into manpages with pod2man
Iterate over libcrypto, libssl and apps
Looks like this has to be repeated for secure/lib/libcrypto, secure/lib/libssl and secure/lib/openssl
Let's try this in secure/lib/libssl
$ cd secure/lib/libssl $ make man-update [Creating objdir /usr/obj/usr/src/amd64.amd64/secure/lib/libssl...] SSL_CIPHER_get_name.3 SSL_COMP_add_compression_method.3 ... SSL_CTX_free.3 SSL_CTX_get0_param.3 Can't open SSL_CTX_get_ex_new_index.pod: No such file or directory at /usr/local/bin/pod2man line 71. *** Error code 2 Stop. make: stopped in /usr/src/secure/lib/libssl
Another OOPS! Let's try man-makefile-update and then man-update
$ make man-makefile-update
rm -f /usr/src/secure/lib/libssl/Makefile.man
echo '# $'FreeBSD'$' >> /usr/src/secure/lib/libssl/Makefile.man
echo '# DO NOT EDIT: generated from man-makefile-update target' >> /usr/src/secure/lib/libssl/Makefile.man
for i in ssl; do for j in /usr/src/crypto/openssl/doc/$i/*.pod; do test -f $j || continue; fn=`basename $j .pod`; if ! echo 'SSLeay_version des_modes' | grep -qw "$fn"; then echo "MAN+= $fn.3"; fi; done; done | env LANG=C sort
>> /usr/src/secure/lib/libssl/Makefile.man
for i in ssl; do for j in /usr/src/crypto/openssl/doc/$i/*.pod; do test -f $j || continue; fn=`basename $j .pod`; if ! echo 'SSLeay_version des_modes' | grep -qw "$fn"; then perl /usr/src/crypto/openssl/util/extract-names.pl < $j
| awk "/^$fn\$/ { next; } { print \"MLINKS+= $fn.3 \" \$1 \".3\" }"; fi; done; done | env LANG=C sort >> /usr/src/secure/lib/libssl/Makefile.man
Can't open perl script "/usr/src/crypto/openssl/util/extract-names.pl": No such file or directory
(repeat many times)
$ make man-update
DTLSv1_listen.3
OPENSSL_init_ssl.3
...
d2i_SSL_SESSION.3
ssl.3
Conclusion: Part of man-makefile-update that collects .pod files works, part that extracts aliases doesn't.
TODO: Fix
MLINKS+= part of man-makefile-update (hint: process-docs.pl)
extract-names.pl and process_docs.pl
extract-names.pl is the old and process_docs is the new script
extract-names.pl
Looks like all it does is read the =head1 NAME section of the .pod files and extract the comma-separated entries contained.
Example from ASN1_ITEM_lookup.pod:
=head1 NAME ASN1_ITEM_lookup, ASN1_ITEM_get - lookup ASN.1 structures
Results in a Perl hash-table
ASN1_ITEM_lookup ASN1_ITEM_get
NOTE: The manpage here is ASN1_ITEM_lookup.3 so somewhere the perl script filters out that link to itself.
This should lead to a single sym-link ASN1_ITEM_get.3 -> ASN1_ITEM_lookup.3
process_docs.pl
Looks like the part we're interested in is line 169 onwards. This uses a feature from OpenSSL::Util::Pod which isn't directly accessible.
An alternative in sed is
sed '/=head1 NAME/,/=head1 /! d;/=head1.*/d;/^\s*$/d;s/ - .*$//;s/,//g' BIO_ADDR.pod || tr ' ' '\n'
Diff for secure/lib/libcrypto/Makefile.inc
Includes the changes for the version
Index: Makefile.inc
===================================================================
--- Makefile.inc (revision 335034)
+++ Makefile.inc (working copy)
@@ -3,7 +3,7 @@
.include <bsd.own.mk>
# OpenSSL version used for manual page generation
-OPENSSL_VER= 1.0.2o
+OPENSSL_VER= 1.1.0h
OPENSSL_DATE= 2018-03-27
LCRYPTO_SRC= ${SRCTOP}/crypto/openssl
@@ -106,7 +106,8 @@
test -f $$j || continue; \
fn=`basename $$j .pod`; \
if ! echo '${_skip}' | grep -qw "$$fn"; then \
- perl ${LCRYPTO_SRC}/util/extract-names.pl < $$j | \
+ sed '/=head1 NAME/,/^=.*/! d;/^=.*/d;/^\s*$$/d' $$j | tr '\n' ' ' | \
+ sed 's/ - .*$$//;s/,/ /g;s/ / /g' | tr ' ' '\n' | \
awk "/^$$fn\$$/ { next; } \
{ print \"MLINKS+= $$fn.${_sec} \" \$$1 \".${_sec}\" }"; \
fi; \