BSD-licensed Text-Processing Tools

Merge the latest version of mdocml (ISC licensed) from the ports into the FreeBSD system base, and configure it as a replacement for the GNU roff (groff) tool set (this would aditionally involve a rewrite of man). There are FreeBSD replacements for sort and diff in ports (also from a GSoC project), but they are lacking features from their GNU counterparts. I will pick up and complete these, possibly borrowing/porting from one of the other BSD projects when possible. Benchmarking the new tools and ensuring they scale appropriately when compared to their GNU counterparts is highly desired, and I would like to try to confirm the mdocml author's assertion that it should be noticeably faster than groff.

A feature-complete (including wide character support) BSD-licensed replacement for:

• man, diff, sort, sdiff, diff3
• Integration of mdocml into FreeBSD base, completely replacing groff
• Integration of above tools into FreeBSD base
• Benchmark comparisons of above tools to previous implementations

• Identify use of man, diff, sort, sdiff, or diff3 in make scripts, build world, etc.
• Performance analysis toolset

• May 24 - May 26: Licensing/Cost Analysis
  • Identify existing tools with a BSD-compatible license (ISC, 2/3/4 Clause BSD, Public Domain?). Contact respective authors about possibility of changing existing code to 2 Clause BSD licensing.
  • Brief analysis for above candidates of effort required to port existing code to FreeBSD versus scratch re-write.

• June 1 - 12: diff
  • (vacation ends)
  • complete diff replacement/port

• June 13 - June 15: diff Testing
  • Verify feature compatibility
  • Exhaustive debugging
  • Performance-measuring, optimization if necessary

• June 16 - June 21: sdiff
  • Complete sdiff replacement/port
• June 21-25: sdiff Testing
  • Verify feature compatibility
  • Exhaustive testing (mergemaster)
  • Performance-measuring, optimization if necessary

• June 26 - July 16: mdocml
  • Merge mdocml (mandoc) into base.
  • Work with Ulrich Spörlein to prevent duplicate work
  • Identify and replace complement scripts with option of compiling man pages with mandoc tools
  • Find and recompose any pages not compatible with mandoc
    
  • GSoC Mid-term *

• July 17 - July 19: man
  • Complement http://people.freebsd.org/~gordon/man.sh (broken link) man replacement
  • Ensure compatibility with mandoc

• July 20 - July 26: sort
  • Complete sort replacement/port
• July 27 - July 30: sort Testing
  • Verify feature compatibility
  • Regression tests
  • Performance-measuring, optimization if necessary

• July 31 - August 7: diff3
  • Complete diff3 replacement/port
• August 7 - August 9: diff3 Testing
  • Verify feature compatibility
  • Regression tests
  • Performance-measuring, optimization if necessary

• August 9 - August 16: Closing Time
  • Documentation
  • Compile performance presentation
  • Test, test, test

==Status Report==

Base version ported from OpenBSD

Base version ported from OpenBSD

Complemented Ulrich Spoerlein's progress: system makefiles have been modified to build mandoc instead of groff and use Gordon Tetlow's modified man.sh when WITH_MANDOC specified in /etc/make.conf .

• Complete patch system for changing mandoc-incompatible man pages.
• Set up a ports package for groff (textproc/groff ?)
• Identify tools with remaining groff dependencies (catman, whatis, …) (looks like Gordon has since done work on this)
  • Finish gordon's man.sh by removing dependencies on groff tools: (eqn, pic, refer)
• Identify remaining mandoc-incompatible man pages. Patch.