pkg_improved [FreeBSD WIKI]

This is an old revision of the document!

The current pkg-tools uses some ineffective ways of keeping track installed packages and it's information, so the goal is to make a self contained project replicating the original pkg_* programs with the use of the BerkeleyDB as a cache for making certain operations a lot faster. Adding additional fields for the package install date and a more useful size information (human readable) could also be needed.

Determine what information should be indexed in the database; The things that are beeing indexed are files installed by each package, it's origin and installdate. By indexing each package the 'pkg_info -W' command executes much faster. By indexing it's origin, origin lookups work a lot faster, and this is used throughout the tools. And by indexing the installdate, the new features in pkg_{info/delete} comparison-search with installdate (e.g., pkg_info -M '*>2008-08-15') works fast.
Implement database support functions (lib); Functions are beeing continually added/removed as needed to make the queries easy and the database-layer abstract so that switching databases if needed, could be made easy for the future.
Modify existing tools to make use of the database cache; So far pkg_info has been improved by using the cache, like the -W option, this is a work in progress and I'm planning to make better size queries, and perhaps adding more fields to the database (like install date, this would have to be discussed).
Write and add a converter(pkg_convert) that caches /var/db/pkg into a database file so that you can make use of the cache
Update the database indexes correctly in pkg_add/pkg_delete and 'make install' in /usr/ports, so that you won't have to run pkg_convert after each add/delete.
Change the output formatting of some of pkg_info's options, especially size and pkg_add's -r option.
Identifying bottlenecks with profiling tools like gprof.

If you have any suggestions or disagreements please contact me at andenore@FreeBSD.org

Last updated: 2008 07 15

Converter:

Added convert program that indexes information to PKG_DBCACHE_FILE
Added some features to lookup keys and print them (or print whole db)

Info:

Improved performance for -W option using the dbcache
Profiling showed that the default behavior for pkg_info was to read plist everytime even though not needed. Added check for this and speed improved significantly.
Uses human readable output for -s (size option) I'm not sure if this breaks things, but it looks Ok. (The old output is available via the -b option)

Add:

Indexes information to dbcache according to the add
Added percentage progress for remote fetching ( -r option )
Fixes dependencies when installing (scans installed ports, checks @pkgdep and adds matching dependency to the installing package's +REQUIRED_BY file)

Delete:

Deindexes information according to the delete

Lib:

Changed methods in match.c to use cache if available (matchallbyorigin, ispkginstalled)
Added global variable 'database' of type DB* (the database object)
Added database helper functions
Added function to cache a Packagelist (cache_plist())
The environment variable PKG_DBCAHE_FILE can be set to make the file location different than the default /var/db/pkg/pkgcache.db

Create:

The -O option has been modified to cache information when installing ports from /usr/ports/

Packinglist:

A @comment DATE:seconds-since-epoch has been added (affects add/create/info/lib)

If you would like to test the improved pkg_install tools here's what's to do:

Download the patch for your release -RELENG_7 or -CURRENT into '/usr/src/usr.sbin/pkg_install/'
cd to '/usr/src/usr.sbin/pkg_install'
Apply the patch with 'patch -p1 < the_patch.diff
Run 'make install' to install the tools.
Run pkg_convert with no arguments to build the cache ( './convert/pkg_convert' )
After this you can use the programs like before.