The HylaFAX "How-To" Guide

Lee Howard,

v4.2.2, 19 Oct 2005

This is a detailed guide to HylaFAX installation, configuration, and troubleshooting.

2. Installation

Please try to use the latest release of HylaFAX software. Sometimes binary-only packages popular in various UNIX distributions are not kept updated. In such cases it is recommended that you uninstall the binary-only package and install HylaFAX from source.

2.1 Modems

This document will not discuss modem installation except to say that the modem must be installed prior to HylaFAX installation. In most cases if minicom, tip, or cu can ''talk'' to it with ATI commands then the modem is installed properly for HylaFAX use.

Selecting a Modem

Really, this is all up to you and your intended use of the faxmodem with HylaFAX. In general, most users are going to fare better using Class 1/1.0 than using Class 2/2.0/2.1 with recent versions of HylaFAX. Even if you're going to try using Class 2/2.0/2.1 it is wise to look for a modem that also supports Class 1/1.0, because if you find a bug in the Class 2 firmware, then you likely will need to rely on the manufacturer for a fix (and experience has shown that this can be difficult to obtain). If the modem supports Class 2 (and you intend to use Class 2) make sure that it supports both 1-D and 2-D image compression and perhaps even 2-D MMR image compression. If the fax line is likely to be busy and resources are limited, then you may be wise to select a modem that supports V.34-Fax (also called SuperG3) which allows faxing at speeds up to 33,600 baud (MultiTech 5634-V92-series, MainPine RockForce, and Eicon Diva Server modems are known to support this).

There is rightfully some concern as to which modem models work (or work best) with HylaFAX. The answer to that question is difficult and will undoubtedly be influenced by one's own experience. However, history on the hylafax-users mailing list has shown that modems with an older Rockwell chipset (RC144DPi, RC288DPi, or K56 found on many various ISA, external serial modems, and Comtrol RocketModems), modems with a newer Conexant chipset (found on MultiTech modems, Comtrol RocketModem IIs, Equinox MultiModems, and Linuxant-driven Linmodems), or modems with a new Lucent/Agere chipset (a.k.a. "Venus" found on MultiTech, Zoom, and MainPine modems) all work very well in Class 1 and 1.0 (if the modem reports support for it via the AT+FCLASS=? command). The Eicon Diva Server is known to work well in Class 2, and the MultiTech 5634-series modems are known to also work well in Class 2.0/2.1 in addition to Class 1/1.0 (especially when using newer firmwares).

As for things to avoid, shipped firmwares for Digi modems will not work in Class 1, and, although functional, the Class 2/2.0 implementation has some known bugs. Also in the functional-but-buggy category are USR/3Com modems; they tend to work better in Class 1 than in Class 2.0, but it still isn't ideal. (USR doesn't seem to be fixing these things, either.) If you use one of these modems and experience problems do not be surprised to learn that it is an issue with the modem itself.

As for anything else, HylaFAX has a fair number of pre-made configuration files for many common modem types. (Modem type is determined by ATI0 and ATI3 commands in Class 1, AT+FMFR? and AT+FMDL? commands in Class 2, and AT+FMI? and AT+FMM? commands in Class 2.0 and 2.1.) However, if you're willing to work on a configuration file, then almost any faxmodem should generally work. Note that modems can vary greatly between model numbers and firmware revisions.

2.2 Source Installation

Tested Operating Systems
AIX 4.1, 4.2* Caldera OpenServer 1.3, 2.3, 2.4* Corel Linux
Debian Linux 2.2 FreeBSD 4.1, 4.2 HP-UX
IRIX* Linux Mandrake 7.1 Linux PPC
Mac OS X* OpenBSD* RedHat Linux 6.x, 7.x
SCO UNIX 3.2v4.2, 4.2, 5.0.4* Slackware Linux 3.5, 7.1* Solaris 7, 8*
SunOS* SuSE Linux 6.x, 7.x, PPC SVR4*
Compaq Tru64* Turbo Linux 6.0 Ultrix*
UnixWare 2.03, 2.13, 7.1.0 YellowDog CS1.1, CS1.2

2.3 Binary Installation

If HylaFAX binaries are available for your environment, many times installing them will prove to be much easier than performing a source installation. Also, binary installations generally will provide a more uniform installation layout similar to other packages already on the system. A binary installation generally maintains the integrity of the system's package management system, especially in the case of RPM, which gives good vigilence over conflicts, requirements, and makes upgrading or uninstallations simple. So, if current HylaFAX binaries are available for your environment, it is generally recommended to use them or make them yourself for installation.

Note, however, that when using binaries made by others that you are often required to use specific versions of dependant packages (i.e. libtiff or awk) based on the packager's configuration.

You should already be familiar with how to install new packages on your system. If not, please consult your system package documentation. The packager should provide installation and configuration instructions that are specific for your system that may vary from the source-installation instructions above.

2.4 SVN Source Installation

If you have an SVN client, and if you wish to be on the cutting/bleeding edge of HylaFAX progress or participate in testing or development, then you can use SVN source from which to build HylaFAX.

The source code is browsable and details are available at

2.5 About JPEG (Color Fax) and JBIG compression support

The usage of JPEG and JBIG compression schemes are generally considered "advanced" fax features and may not be available to all fax machines. JPEG compression is most commonly used for sending and receiving color facsimile. JBIG compression is most commonly used for sending and receiving monochrome facsimile, but with a tighter compression (and thus faster send-times) than MMR. Both JPEG and JBIG facsimile compression require the usage of error-correction protocol (ECM).


HylaFAX+ 5.4.1 and later versions support both sending and receiving of color facsimile (JPEG).

HylaFAX JPEG support is dependent upon the presence of libjpeg, the presence of LittleCMS, and libtiff v4.0.0 and later.

When compiling HylaFAX, you should see the following messages during ./configure if you expect color fax support:

    Checking for JPEG library support
    ... found. Enabling JPEG library support
    Checking for Little CMS library support
    ... found. Enabling Little CMS support

Then after building and installing HylaFAX, color fax reception support can be enabled in the modem config file by setting:

    Class1ColorJPEGSupport: yes
Submit color faxes with sendfax like this (where 1235 is the number to dial and color.pdf is the color document to send):
    sendfax -O usecolor:yes -d 12345 color.pdf

This "usecolor:yes" tells HylaFAX+ to prepare both monochrome and color documents. If the receiver supports color then the color document will be sent. Otherwise, the monochrome will be sent as traditional.

If you want to send a color fax and ensure that it's only sent in color (i.e. fail the fax job if the receiver doesn't support color) then use this syntax:

    sendfax -O usecolor:only -d 12345 color.pdf

HylaFAX+ uses Ghostscript to convert all color documents into non-compressed 24-bit RGB data. So Ghostscript will produce a big file in the document preparation phase, and you'll see this big file sitting in your docq directory. HylaFAX+ faxsend will take that non-compressed data and will compress it into JPEG just after it images the tagline onto the page during transmission. It does it this way to avoid multiple compression+decompression steps with JPEG data. (Remember that JPEG is lossy and compressing and decompressing and compressing again causes loss of detail.)

Receiving a fax should work just as expected. Because fax machines use ITULAB colorspace which is not commonly supported in viewer applications, HylaFAX+ converts received JPEG faxes into YCbCr colorspace. (So yes, there is some loss in detail, but it is minimal.) You should then be able to view the received TIFF in any TIFF viewer application... or as is popular, you should be able to convert it to PDF with libtiff's tiff2pdf tool. Be aware, however, that libtiff's tiff2ps tool currently does not work correctly for this. So if you need received faxes in Postscript use tiff2pdf and then use Ghostscript's pdf2ps.

Recognize that color images are much, much larger than monochrome images. Consequently communication times for color faxes is significantly greater than for monochrome. Watch this. Some senders may inadvertently push the "color send" button on their fax machine even when they are only transmitting black-and-white text. The result will be a very long-winded fax for no benefit.


HylaFAX supports both sending and receiving JBIG-compressed facsimile in Class 1/1.0. The Class 2.1 specification does not include JBIG, and so JBIG is not likely to be available with Class 2/2.0/2.1.

Like JPEG, HylaFAX JBIG receive support is entirely dependent upon JBIG support being available in libtiff. Most recent libtiff releases include the ability to write JBIG-compressed TIFF files. However, doing anything with the TIFF file from there will be difficult as TIFF viewers usually do not support JBIG compression. So usually the JBIG-compressed TIFF file requires conversion to PDF or a non-JBIG compression for viewing. This will require the use of the JBIG-KIT library by libtiff's tiff2pdf tool.

At this point, then, JBIG fax reception support should be enabled by default but can be manipulated in the modem config file with Class1JBIGSupport.

HylaFAX JBIG send support utilizes the ModemSoftRTFCC feature. Thus, sending JBIG-compressed facsimile requires that ModemSoftRTFCC be enabled (which is the default setting). If the JBIG-KIT libraries were found when HylaFAX was compiled then the ModemSoftRTFCC feature will also convert to JBIG when appropriate (and Class1JBIGSupport will also default to "yes"). JBIG send support does not require libtiff support.

2.6 The Need for Real-Time Operating System Support

Traditionally UNIX (including Linux) has attempted to treat all processes, users, and hardware drivers fairly such that no one of them could starve the others from attention. This approach works well for many types of network applications and other server-related purposes where the need for any one process, user, or driver to have undisputed real-time attention of the system resources does not usually exist. In the traditional UNIX scenario if some user runs a program that could potentially consume all of the resources of the system (for example, a program in an infinite tight loop eating up CPU), the administrator can still eventually log in and perform operations, and other processes are not entirely starved of that same resource (although, perhaps responsiveness is decreased). This is intentional.

However, there are some real-time applications where this operating-system fairness can be detrimental to application performance. Some examples are audio recording/playback, telephony services, and [yes!] fast serial communications (including serial communication to fax modems).

Typically, if you're using a Class 1 modem (where the amount of data going between the modem and the host is done in relatively small chunks) and the DTE-DCE bitrate (ModemRate in HylaFAX modem config files) is low (e.g. 19200 bps) then you're not likely to have problems. However, as the communication bitrate is increased (say, to 115200 bps) and the amount of data being communicated increases (such as a 64 K block of image data from a Class 2.1 modem in ECM mode) the likelihood of some other system activity (such as medium to heavy hard drive or network activity) interrupting the communication is also increased.

In those cases what happens is that the system tries to treat the hard drive activity equal with the serial modem activity, and in so-doing the system can actually fall beind on getting data from the serial modem, and thus data gets lost. Missing data from an MMR image will cause image truncation, but everything will look normal and proper in the HylaFAX logs and to the modem itself. In fact, in those scenarios the system doesn't even know that it missed data, and the only way to know that data did get lost is by looking at the data (and perhaps comparing it with the original).

As mentioned, this problem also is an issue for telephony applications such as Asterisk using real-time hardware interfaces such as zaptel-driven cards. It's typically very easy to spot in the zaptel case by using the zttest diagnostic tool. If zttest does not report 100% then it typically means that the IRQ for the zaptel card has a lower priority than some other hardware (such as a hard drive) or that the kernel (and this can vary greatly between kernel versions) is not honoring the interrupt priority of the zaptel hardware (again, probably because it's trying to be fair to other lesser-priority hardware). In the zaptel case it causes audio to go missing, and if that telephony device is being used for faxing (such as with IAXmodem) then the missing audio can cause image data corruption or retransmissions. This can lead to high error rates.

The answer to the problem is, unfortunately, not a simple one - particularly on Linux. The general solution is to alter the system configuration such that the operating system (i.e. the kernel) prioritizes the hardware expecting real-time support. Sometimes this can be acheived by hardware configuration (i.e. seeing that high-priority devices are assigned high-priority IRQs and that the kernel honors those priorities - and the kernel's honoring of that priority can be hit-and-miss depending on the particular version). Sometimes this can be acheived with driver tuning... i.e. by adjusting DMA settings on hard drives or by using various software tricks (in the Linux 2.0/2.2 days there was an "irqtune" program that may have helped). However, in many cases and probably the only realiable long-term general Linux solution is to get a kernel with "real-time" support. Some Linux vendors sell real-time Linux versions. These should work. However, real-time Linux kernels are available freely through patches provided by the Linux "rt" project.

Here is an example of how real-time Linux can be installed.