Gentoo Portage secrets

Friday August 11, 2006 (08:01 AM GMT)

By: Aleksey Alekseyev

Gentoo Linux is perhaps the most-used source-based Linux distribution. One secret to its success is the powerful and handy Portage package management system. While Gentoo comes with extensive documentation covering most aspects of using Portage, the techniques described in Gentoo's handbook and other documentation are not always the most effective ones. Here are some insider tips that can greatly increase your productivity.



Search faster

Before you install a package, you usually look for it via Portage's search capabilities. Portage's emerge utility has --search and --searchdesc options, but using them is not enjoyable, because they take a long time to run. That's why we've seen the emergence of third-***** search front ends for Portage, such as esearch and eix. Their common idea is to use their own search indexes to speed up searches. When using either utility, you have to rebuild the index after updating the Portage tree, and after installing and uninstalling software.

Of the two, eix works faster and has more capabilities. You can get information on the utility from its man page or by invoking eix --help. To use eix to search for a package whose name contains foo, simply invoke eix foo.

Eix is a very flexible tool. It can give you more information on packages than esearch or emerge -s. It can search through different fields (e.g. package name, category, or description), it can search for regular expressions or wildcard patterns, or do fuzzy searches, and its output can be configured for use in scripts.

Optimizing traffic usage

Updating your software can take a lot of network bandwidth. There are tools that help you decrease Portage's appetite. The most effective and well-known such tool is Deltup, which allows you to download deltas, or the differences between new and old versions of package source. That approach can save you up to 90% of the download size. The procedure for installing deltup is described in the Gentoo wiki.

Using Deltup has its drawbacks, however. Deltup will not resume fetching a delta if you lose a connection. Sometimes, the server generating the deltas is overloaded, and you must wait for a long time for it to generate your delta. Sometimes Deltup gets confused about which versions of packages you have. And, of course, generating a new package takes some CPU time. Still, Deltup should not break your system, so you should not be afraid to give it a try.

KDE maintainers generate the deltas for their packages themselves. This HOWTO describes using these deltas in Gentoo. There was a period when Portage supported a kdexdeltas USE flag, so you did not need to patch your sources manually, but after KDE 3.5 this functionality was not updated, so the kdexdeltas flag will not give you any advantage these days. Note that if you receive errors that report "digest verification failed" when using manually updated KDE sources, you can still install new KDE packages by using the emerge --digest command.

You can use the power of deltas not only when updating source packages, but also when updating your Portage tree. Emerge-delta-webrsync allows you to download daily patches to the Portage tree, which is less traffic-consuming than using emerge --sync. Just invoke emerge emerge-delta-webrsync, and next time you need to sync your Portage tree, run emerge-delta-webrsync

The Gentoo wiki provides more tips for users with poor Internet connections.

A new feature that came in Portage 2.1 can also help speed downloads. If you add the string parallel-fetch to the FEATURES variable in /etc/make.conf, emerge will download some packages' source code while compiling others that are in the queue. This reduces installation time, especially if you emerge many packages.

Faster package compilation

Every time you install or update some software under Gentoo, you need to wait until its source is compiled. Portage supports a set of tools that try to decrease the compiling time of your packages.

One of the ways to speed up compilation on a slow machine is to distribute the compiling task to another host. This is what Distcc aims to do. You can even use Windows boxes to assist in the task.

Ccache makes it possible to speed up compilation of the same code by caching compilation results. Confcache is a similar tool for caching results of test configuration scripts (./configure). It appeared at the same time as Portage 2.1, and still lacks full documentation, but it has been mentioned in the Gentoo Linux Newsletter. After spending some time in the unstable branch, it is now hardmasked, which means that confcache has some unresolved issues; if you try using the current confcache version (0.4.2) you will sometimes need to install some packages with the confcache feature turned off, or clean the confcache cache (rm -rf /var/tmp/confcache).

All the tools mentioned in this section can be used for any compilation tasks. See the appropriate documentation for Distcc, ccache, and confcache (coming later).

Managing logs and configuration files

Another new feature in Portage 2.1 is the new logging framework. Many packages show notices while you emerge them, but you may not see them if you do not watch the emerge process. Now you just need to turn on elog, by adding these lines to /etc/make.conf:
# This sets what to log
PORTAGE_ELOG_CLASSES="warn error log"
# And this is how to do it
PORTAGE_ELOG_SYSTEM="save"
and creating the /var/log/portage/elog directory. Now, each package's emerge log will be saved in a separate file in the specified directory. You can find more info on configuring elog by inspecting the /etc/make.conf.example file. There are already GTK and QT graphical front ends for viewing these logs.

After updating some packages you sometimes need to update their configuration files. The standard tool for this is etc-update, but there is a more handy and advanced utility -- dispatch-conf. It automatically updates configuration files that have easy changes to comments, or that have not been changed between updates. It also allows you to redo any changes you made to configuration files by placing them into a version control system.

Gentoo Portage 的秘笈
2006年8月11日，星期五（08:01AM GMT）
作者：Aleksey Alekseyev

Gentoo Linux或许是最多用户使用的基于源代码编译的Linux发行版。它成功的一个秘密就是强大而便利的Portage包管理系统。当Gentoo拥有大量各方面的文档，包括许多关于如何使用Portage的文档，那些在Gentoo手册以及其他文档中讲述的技术已经并不是最有效的了。这儿有一些内部的小技巧将可以极大的提升你的“生产率”

令搜索更迅速

在你安装一个包之前，你常常会通过Portage去查找它。Portage的emerge工具拥有--search和--searchdesc两个选项，但使用他们并不这么令人愉快，因为他们将花费很长时间去运行。这就是我们要寻找第三方的Portage查找前端的原因，例如esearch和eix。它们共同的思想是使用他们自己的查找索引使查找效率提高。当使用这些工具的时候，你必须在升级了Portage树，或者安装及卸载软件之后重建索引。

在这两个软件中，eix的速度更快，效率更高。你可以从man page或执行eix --help获得关于这个工具的信息。例如，使用eix去查找一个名字中带有foo的软件包，只需要简单的执行eix foo。
Eix是一个非常灵活的工具。它可以为你提供比esearch或者emerge -s更多关于软件包的信息。它可以查找不同的属性（如软件包名，分类，或者描述），也可以使用正则表达式，通配符，或者进行模糊查询，而且它的输出还可以设置好被脚本利用。（汗，这句好别扭，不知道对不对= =）


第二部分的delta是什么意思？汗，对gentoo的了解还是太少了。。。

增量， 差别

优化网络使用

升级你的软件会占用相当多的网络贷款。这儿有一些工具帮助你降低Portage的“欲望”。最有效而且有名的工具叫Deltup，它允许你下载“deltas”（增量的部分），或者新旧不同版本源代码中不同的部分。这将节省你将近90%的下载量。安装Deltup的方法在Gentoo wiki有介绍。

当然，使用Deltup也有缺点。Deltup在你丢失和服务器连接的时候，不会恢复下载delta。有时候服务器生成的delta被覆盖了，而你必须等待相当长的时间直到它生成你的delta。有时候Deltup会对你现有的软件包版本产生混乱。而且，显而易见的是生成一个新的软件包将占用一点CPU时间。然而，Deltup将不会破坏你的系统，所以你不需要担心去尝试它。

KDE维护者会生成他们自己的delta。这篇HOWTO描述了如何在Gentoo使用这些delta。Portage支持kdexdeltas的USE flag已经有一段时间了，因此你并不需要为源代码手动打补丁，但在KDE3.5只有，这个功能并没有升级，所以现在kdexdeltas的USE flag并不能给你带来任何好处。请注意当你因为手动升级KDE源代码而收到"digest verification failed"的错误提示时，你依然可以通过emerge --digest命令安装新的KDE包。

你并不仅仅可以在升级源代码包时候使用deltas的强大功能，同样可以在升级Portage树时使用到它。Emerge-delta-webrsync允许你为Portage树下载日常补丁，它将比emerge --sync占用更少的带宽。只需要执行emerge emerge-delta-webrsync，当下次你需要更新Portage树的时候，执行emerge-delta-webrsync

Gentoo wiki提供了更多关于低带宽Internet用户的小技巧。

Portage 2.1的一个新特性同样可以加速下载。如果你在/etc/make.conf的FEATURES变量中加入parallel-fetch，emerge将在编译软件包时下载一些队列中其他软件包的源代码。这将减少一些安装时间，尤其是你emerge很多软件包的时候。

加速软件包的编译

每当你在Gentoo中安装或升级一些软件的时候，你需要等待它的代码完成编译。Portage支持一系列试图减少软件包编译时间的工具。

其中一个在速度较慢的机器上加速编译的方法是将编译任务分配给其他主机。这正是Distcc的目标。你甚至可以使用Windows主机来帮助完成任务。

Ccache使用缓存编译结果来加速编译同样的代码。Confcache是一个类似的工具，它缓存设置脚本（./configure）的结果。这同时出现在Portage 2.1中，虽然目前仍然缺乏完整的文档，但已经在Gentoo Linux Newsletter中提及。在一段时间的unstable分支测试之后，它现在被hardmask，意味这confcache还有一些关键没有解决。如果你试图使用当前版本的confcache(0.4.2)，你在安装一些软件包的时候需要关闭confcache特性，或者清理confcache的缓存（rm -rf /var/tmp/confcache）。

所有在这部分提到的工具可以在所有编译任务中使用。详细请查阅关于Distcc、ccache和confcache(coming later)的文档

管理日志和设置文件

另一个Portage 2.1中的新特性是新的日志记录框架。许多软件包在emerge的时候显示notice消息，但如果你并不注意emerge过程的话，你或许并没有看到他们。现在你只需要加入这几行到/etc/make.conf

# This sets what to log
PORTAGE_ELOG_CLASSES="warn error log"
# And this is how to do it
PORTAGE_ELOG_SYSTEM="save"

打开elog，并创建/var/log/portage/elog目录，这样每个软件包的emerge日志将被存储为/var/log/portage/elog目录中独立的文件。你可以在/etc/make.conf.example文件中找到关于elog设置更多的信息。同样有查看那些日志记录的GTK和QT图形前端软件。

在升级完一些软件包后，你有时候需要更新它们的设置文件。完成这个任务的标准工具是etc-update，但这有一个更容易操作而且更强大的工具——dispatch-conf。它将自动更新一些含有简单的备注改变或者并没有改变的配置文件。它通过版本控制系统，同样允许你恢复任何对于设置文件的改变。

starain，辛苦啊
翻译人才啊

不知道starain有没有兴趣参与Gentoo文档的翻译
现在这个项目还未开始（确切的说是reload）
如果有兴趣，请先订阅http://groups.google.com/group/gentoo-china/

谢谢两位夸奖，注册N久以来一直都是在linuxsir潜水学习，见到有力所能及的事情就做点小贡献了：）
好的，我订阅看看学习一下。

Excellent