=========================
 DEBPARTIAL-MIRROR IDEAS
=========================

This page is to coordinate `debpartial-mirror`_ development.

.. _`debpartial-mirror`: http://packages.debian.org/unstable/net/debpartial-mirror

Introduction
============

``debpartial-mirror`` is a tool to create a local partial mirror of
apt-repository.

The *partial-mirror* can be used both as a source of packages to build
**Custom Debian Distributions** archives, cd-images or local APT
repositories.

The packages included into the partial mirror can be downloaded from
remote APT repository or local directories. 

The list of packages that are included into the partial-mirror is
specified by means of a configuration file: it is also possible to
filter from a package list.

Mirror organization
-------------------

The partial mirror is organized as a collection of Backends. A Backend is a
repository with packages from one source (a local or a remote repository):

  /partial_mirror/
    sarge/
    sid/
    debian-np/
    local-np/
    br/
    it/
    ....

In turn each backend, has a fixed structure:

  /partial_mirror/backend/
    dists/
    pool/
    
the ``dists/`` directory contains the ``Packages.gz``, ``Sources.gz`` and
``Release`` files for the Backend, while the ``pool/`` directory contais an
apt pool-style tree of packages.

Merged backends
---------------

Once each Backend is filled, it is possible to create several *Merged
Backends*.

A *Merged Backend* is a Backend built using a list of Backends from the local
mirror that contains a ``dists/`` dir that merges the contents of the
different ``dists/`` subdirectories of the selected Backends and a ``pool/``
subdirectory built using symbolic (or hard) links to the packages included on
the merged Backends.

In addition, when the Merged Backend is created, it is possible to filter
which packages to include from the selected Backends.

Backend snapshots
-----------------

Each backend has a list of binary and source packages inside the ``dists/``
subdirectory and a copy of the corresponding files into the ``pool/`` dir.

The standard way of updating each backend would be to download the new
Packages and Sources lists, remove the old files from the pool and download
the new ones.

A very interesting option could be to support multiple *snapshots* of the same
backend, the idea would be to keep diferent versions of the ``dists/``
subdirectory using a file structure like the following:

  /partial_mirror/backend/
    snapshots/YYYYMMDD/dists/
    snapshots/YYYYMMDD+1/dists/
    ...
    dists/
    pool/

With this schema each ``dists/`` subdirectory contains the corresponding
lists of source and binary packages and the ``pool/`` subdir contains the
source and package files of all included snapshots.

Of course the system has to allow the selective removal of one snapshot (i.e.,
remove the 20041109 snapshot), that is, it has to support the removal of  the
source and package files that belong to the selected snapshot and are not
included on any of the rest of *snapshots*.

Shared pool
-----------
Extending the idea of the *snapshots* presented before we could use a shared
``pool/`` to store the source and package files included on each *backend*.

This could reduce the download time and the pool size; if a file is included
into two *backends* it is only installed once.

The problem with this model is similar to the one described earlier: to remove
unwanted files from previous *backends* without breaking all the rest we have
to calculate the differences as described when talking about removing a
*snapshot*.


Program description
===================

.. Note: This is different from the actual implementartion, the following
   could be used as trace for the man-page.

SYNOPSIS
........

::

  debpartial-mirror [-c=file] 
    {update [backend, [....]] | merge [backend, [....]] |
    | check | clean [backend, [....]]}

DESCRIPTION
...........

``debpartial-mirror`` will accept some command in the way apt-get does, and
read conf information from a configuration file (default
``/etc/debpartial-mirror/debpartial-mirror.conf``) that in turn can include
other conf files.

update

  this command let debpartial-mirror to update the backends that
  compose the partial mirror. If no backend is specified, all
  backends will be update. Else, only the listed backends will be
  updated. 

merge

  this command will create one (or more) merge backend. The list
  of backends to be used to merge is specified in the main
  conf-file.

After update and/or merge, debpartial-mirror will clean up the mess
removing old and uneeded files from it's structure.

Notes about the source code
===========================

The backend class
-----------------

Properties:
	need_update (boolean)

Members:
	Dists -> A class that realize backend/dists
	PackageList -> A class that contains the list of packages that
		       should be copied into the partial-mirror of this
		       backend 

Methods:
	create-> 
	update -> Sync the partial-mirror with the sources repositories
	__create_package_list -> Generate a package list reading from
				 main conf-file.

The dists class
---------------

Properties:
	got_files
        
Methods:
	update -> Update the Dists files (Packages, Release and Sources)
	_get -> Get each file
