The current pkg-tools uses some ineffective ways of keeping track installed packages and it's information, so the goal is to make a self contained project replicating the original pkg_* programs with the use of the BerkeleyDB as a cache for making certain operations a lot faster. Adding additional fields for the package install date and a more useful size information (human readable) could also be needed.

• Determine what information should be indexed in the database; The things that are beeing indexed are files installed by each package, it's origin and installdate. By indexing each package the 'pkg_info -W' command executes much faster. By indexing it's origin, origin lookups work a lot faster, and this is used throughout the tools. And by indexing the installdate, the new features in pkg_{info/delete} comparison-search with installdate (e.g., pkg_info -M '*>2008-08-15') works fast.
• Implement database support functions (lib); Functions are beeing continually added/removed as needed to make the queries easy and the database-layer abstract so that switching databases if needed, could be made easy for the future.
• Modify existing tools to make use of the database cache; So far pkg_info has been improved by using the cache, like the -W option, this is a work in progress and I'm planning to make better size queries, and perhaps adding more fields to the database (like install date, this would have to be discussed).
• Write and add a converter(pkg_convert) that caches /var/db/pkg into a database file so that you can make use of the cache
• Update the database indexes correctly in pkg_add/pkg_delete and “make install” in /usr/ports, so that you won't have to run pkg_convert after each add/delete.
• Change the output formatting of some of pkg_info's options, especially size and pkg_add's -r option.
• Identifying bottlenecks with profiling tools like gprof.

If you have any suggestions or disagreements please contact me at andenore@FreeBSD.org

Converter:

• Added convert program that indexes information to PKG_DBCACHE_FILE
• Features to lookup keys and print them (or print whole db)
• Caches origin, installtime and which-files (files in package to pkgname)

Info:

• Improved performance for -W option using the dbcache
• Profiling showed that the default behavior for pkg_info was to read plist everytime even though not needed. Added check for this and speed improved significantly.
• Uses human readable output for -s (size option) I'm not sure if this breaks things, but it looks Ok. (The old output is available via the -b option)
• Print installation date with -n (human readable) or -N (seconds since epoch)
• Added installtime comparison with the -M option, e.g, “pkg_info -M '*>2008-07-18 18:30:21'” will list all packages installed after the date YYYY-MM-DD hh:mm:ss.

Add:

• Indexes information to dbcache according to the add
• Added progress indication to the remote fetching option (-r), similar to that of fetch(1)
• Fixes dependencies when installing (scans installed ports, checks @pkgdep and adds matching dependency to the installing package's +REQUIRED_BY file)

Delete:

• Deindexes information according to the delete
• Now supports range deletion with installdates, e.g., “pkg_delete -i -M '*>=2008-07-14<2008-08'” will delete all packages installed between the dates 2008-07-14 and 2008-08.

Lib:

• Changed methods in match.c to use cache if available (matchallbyorigin, ispkginstalled)
• Added global variable 'database' of type DB* (the database object)
• Added database helper functions
• Added function to cache a Packagelist (cache_plist())
• The environment variable PKG_DBCAHE_FILE can be set to make the file location different than the default /var/db/pkg/pkgcache.db
• pattern_match() will compare dates as well as version numbers.

Create:

• The -O option has been modified to cache information when installing ports from /usr/ports/

Packinglist:

• “@comment DATE:YYYY-MM-DD hh:mm:ss” has been added to installed packages (affects add/create/info/lib)

Fixed bugs which exists in -CURRENT and possibly earlier versions:

• Fixed a bug with the oldstyle -W handling of empty @cwd's, which leads to a number of files who are installed by packages not beeing recognized.
• pkg_add -r of two packages (or more) where the first depends on one or more packages which is not installed causes a “fatal error during execution: getcwd” because it doesn't save previous working directories correctly.

If you would like to test the improved pkg_install tools here's what to do:

1. Download the latest patch for your release at http://home.no.net/andenore/patches/ into “/usr/src/usr.sbin/pkg_install/”
2. cd to '/usr/src/usr.sbin/pkg_install'
3. Apply the patch with “patch -p1 < the_patch.diff”
4. Run “make && make install” to install the tools.
5. Run pkg_convert(1) with no arguments to build the cache.
6. After this you can use the programs like before.

If you notice any bugs your input will be greatly valued and please contact me.

NB. If you do get this:

duckjen# make
===> lib (all)
"../Makefile.inc", line 3: Cannot open /usr/share/mk/bsd.own.mk
"../Makefile.inc", line 17: Cannot open ../Makefile.inc
make: fatal errors encountered -- cannot continue
*** Error code 1

You are probably not compiling from /usr/src/usr.sbin/pkg_install/ and Makefile.inc will try to .include "../Makefile.inc" so comment out this line and specify BINDIR.

# Inherit BINDIR from one level up.
#.include "../Makefile.inc"
BINDIR?= /usr/sbin

• Patches
• Perforce depot: //depot/projects/soc2008/andenore_pkginstall/
• Blog (a little more info about the progress)
• Contact information here AndersNore