pkg_improved [FreeBSD WIKI]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== pkg_improved =====

==== About ====
The current pkg-tools uses some ineffective ways of keeping track installed packages and it's information, so the goal is to make a self contained project replicating the original pkg_* programs with the use of the %%BerkeleyDB%% as a cache for making certain operations a lot faster. Adding additional fields for the package install date and a more useful size information (human readable) could also be needed.

==== Milestones ====
  * Determine what information should be indexed in the database; The things that are beeing indexed are files installed by each package, it's origin and installdate. By indexing each package the 'pkg_info -W' command executes much faster. By indexing it's origin, origin lookups work a lot faster, and this is used throughout the tools. And by indexing the installdate, the new features in pkg_{info/delete} comparison-search with installdate (e.g., pkg_info -M '*>2008-08-15') works fast.
  * Implement database support functions (lib); Functions are beeing continually added/removed as needed to make the queries easy and the database-layer abstract so that switching databases if needed, could be made easy for the future.
  * Modify existing tools to make use of the database cache; So far pkg_info has been improved by using the cache, like the -W option, this is a work in progress and I'm planning to make better size queries, and perhaps adding more fields to the database (like install date, this would have to be discussed).
  * Write and add a converter(pkg_convert) that caches /var/db/pkg into a database file so that you can make use of the cache
  * Update the database indexes correctly in pkg_add/pkg_delete and "make install" in /usr/ports, so that you won't have to run pkg_convert after each add/delete. 
  * Change the output formatting of some of pkg_info's options, especially size and pkg_add's -r option.
  * Identifying bottlenecks with profiling tools like gprof.

If you have any suggestions or disagreements please contact me at <andenore@FreeBSD.org>

==== Changes ====

Converter:
  * Added convert program that indexes information to PKG_DBCACHE_FILE
  * Features to lookup keys and print them (or print whole db)
  * Caches origin, installtime and which-files (files in package to pkgname)

Info:
  * Improved performance for -W option using the dbcache
  * Profiling showed that the default behavior for pkg_info was to read plist everytime even though not needed. Added check for this and speed improved significantly.
  * Uses human readable output for -s (size option) I'm not sure if this breaks things, but it looks Ok. (The old output is available via the -b option)
  * Print installation date with -n (human readable) or -N (seconds since epoch)
  * Added installtime comparison with the -M option, e.g, "pkg_info -M '*>2008-07-18 18:30:21'" will list all packages installed after the date YYYY-MM-DD hh:mm:ss.

Add:
  * Indexes information to dbcache according to the add
  * Added progress indication to the remote fetching option (-r), similar to that of fetch(1)
  * Fixes dependencies when installing (scans installed ports, checks @pkgdep and adds matching dependency to the installing package's +REQUIRED_BY file)

Delete:
  * Deindexes information according to the delete
  * Now supports range deletion with installdates, e.g., "pkg_delete -i -M '*>=2008-07-14<2008-08'" will delete all packages installed between the dates 2008-07-14 and 2008-08.

Lib:
  * Changed methods in match.c to use cache if available (matchallbyorigin, ispkginstalled)
  * Added global variable 'database' of type DB* (the database object)
  * Added database helper functions
  * Added function to cache a Packagelist (cache_plist(%%))%%
  * The environment variable PKG_DBCAHE_FILE can be set to make the file location different than the default /var/db/pkg/pkgcache.db
  * pattern_match() will compare dates as well as version numbers.

Create:
  * The -O option has been modified to cache information when installing ports from /usr/ports/

Packinglist:
  * "@comment DATE:YYYY-MM-DD hh:mm:ss" has been added to installed packages (affects add/create/info/lib)

Fixed bugs which exists in -CURRENT and possibly earlier versions:
  * Fixed a bug with the oldstyle -W handling of empty @cwd's, which leads to a number of files who are installed by packages not beeing recognized.
  * pkg_add -r of two packages (or more) where the first depends on one or more packages which is not installed causes a "fatal error during execution: getcwd" because it doesn't save previous working directories correctly.

==== Testing ====
If you would like to test the improved pkg_install tools here's what to do:
  - Download the latest patch for your release at http://home.no.net/andenore/patches/ into "/usr/src/usr.sbin/pkg_install/"
  - cd to '/usr/src/usr.sbin/pkg_install'
  - Apply the patch with "patch -p1 < //the_patch.diff//"
  - Run "make && make install" to install the tools.
  - Run pkg_convert(1) with no arguments to build the cache.
  - After this you can use the programs like before.
If you notice any bugs your input will be greatly valued and please contact me.

:!: NB. If you do get this:

<code>
duckjen# make
===> lib (all)
"../Makefile.inc", line 3: Cannot open /usr/share/mk/bsd.own.mk
"../Makefile.inc", line 17: Cannot open ../Makefile.inc
make: fatal errors encountered -- cannot continue
*** Error code 1
</code>

You are probably not compiling from ///usr/src/usr.sbin/pkg_install/// and //Makefile.inc// will try to ''%%.include "../Makefile.inc"%%'' so comment out this line and specify BINDIR.

<code>
# Inherit BINDIR from one level up.
#.include "../Makefile.inc"
BINDIR?= /usr/sbin
</code>

==== Links ====
  * [[http://home.no.net/andenore/patches/|Patches]]
  * Perforce depot: %%//%%depot/projects/soc2008/andenore_pkginstall/
  * [[http://blogs.freebsdish.org/andenore/|Blog (a little more info about the progress)]]
  * Contact information here [[/AndersNore|AndersNore]]