ISPMan Handbook

version 1.0

by Tony Wasson

Last Update: March 10, 2004

Chapter 1: Overview of ISPMan

Letter from Atif Ghaffar (hopefully, someday..)

Components & Architecture

ISPMan is an LDAP based system to manage user accounts, email, web hosting, and access control. ISPMan provides a web management interface to view, edit and change domains, user and email accounts, space quotas, FTP access, and web server settings. ISPMan also features an even more powerful set of command line utilities, so that administrators who prefer the CLI don't need to use a browser.

ISPMan was designed from the beginning for scalability. It has 4 components:

LDAP Directory. ISPMan keeps a central registry of information about users, hosts, dns, processes inside the LDAP directory. The LDAP directory can be replicated to multiple machines to balance the load.

ISPMan web interface is an intuitive Dynamic HTML (requires Javascript) interface to manage information about domains, users and configuration settings. This interface edits the LDAP directory and schedules processes for agents when needed. Depending on access rights, authenticated users can take actions such as add a new domain, delete a user, or reset a password.
The interface runs on http or https and is only available after successful authentification as an administrator, reseller or client. Access control to this interface can also be limited to designated IP addresses.

ISPMan-agent runs on hosts and ensure that the host daemons are configured like the LDAP directory. The ispman-agent, or agent, examines the LDAP directory for processes assigned to them and take appropriate actions
Example: create directory for new domains, create mailbox for users, etc. These agents are a very important part of the system and are should be run continuously.
The agents are run via a fault taulerant services manager called “daemontools” that makes sure that the agents recovers immediately in case of any failure.

The Customer Control Panel is a seperate web interface targeted towards customers and domain owners. Using this interface the customer, or client, can manage their own dns, webserver settings, users, mailing lists, access control etc. It is designed to be simpler than the ISPMan Admin web interface.

Network Examples:

The best way to start undertstanding ISPMan is to see how the system works. This diagram shows all the components working together, along with the external applications.

Single Server

While a single server is not the recommended setup, it is useful to show how all the components fit together. This server will need to run all 4 ISPMan components : LDAP Directory, ISPMan web interface, ISPMan-agent and the Customer Control Panel. It will also need to run external applications to handle DNS, email and web hosting.

All configuration takes place from one of the web interfaces or CLI. ISPMan-agent makes sure that configuration files are up to date, and restarts or reloads external applications as needed.

Typical Installation

A typical installation of ISPMan will have several LDAP servers. These LDAP servers are kept in sync by replication, usually by the program slurpd. There will be a master LDAP server, and the rest will be copies. This setup is useful not only for balancing the request load of each LDAP server, but also for disaster recovery (or avoidance!).

??Figure of medium sized network with LDAP replication occuring.

Large Setup

A large system will not differ dramatically from a typical installation. The main difference will be that there are many servers all running the ispman-agent. Replication is a must for large installations.

??Figure of large distributed system.

Filesystem Layout

ISPMan has 2 main directories it works with, the hosting data (under /ispman) and the application (under /opt/ispman). These are the default locations and may differ depending on your installation or packager's choices.

/ispman/domains

The domains subfolder primarily contains web hosting data. ISPMan-agent needs to be able to create and delete directories here. Each domain gets it's own subdirectory which will contain user and vhost files.

example.com/vhosts – Each vitualhost in ISPMan has it's own subdirectory.

example.com/vhosts/www/htdocs (for content)

example.com/vhosts/www/cgi-bin (for scripts and executables)

example.com/vhosts/www/logs (not used)

example.com/users – This example has just 2 users.

example.com/users/atif_example_com

example.com/users/arno_example_com

/ispman/logs

vhosts (still used?)

vhosts/www.example.com.access

vhosts/www.example.com.error

domains (still used?)

/opt/ispman

bin contains more than 100 command line utilities, administrators never need to use the web interface for day to day tasks

build the “source” is here, use cvs update to build and apply bug fixes and desirable patches

cgi-bin ISPMan admin and customer interfaces rely on perl/HTML templates found here

conf core config file for ISPMan and language defaults

htdocs the ISPMan admin interface and “customer control panel” html

lib contains the perl modules ISPMan is built on

locale alternate languages live here

templates dns, apache, and other configurable templates are here.

var logs of the ispman-agent live here

Processes

ISPMan schedules system changes using processes. Processes are steps to be done by ispman-agent running on each machine. A few examples of processes are adding an A record to a domain, or adding a mailbox for a user. Processes are directed to hosts based on hostgroup membership. You can view the Processes "in the queue" by clicking on the Processes tab at the top of the screen, or using the CLI command 'ispman.showprocesses'.

A process can have the status of "new" or "insession" or “error”.

• insession - the process is waiting for the user to click "Commit session". Clicking Commit session changes the state to new.

• new - the process is waiting for the ispman-agent to execute it.

• error – ispman-agent was unable to complete the process. Check the agent log file.

If the user logs out of the web interface before clicking "Commit session" the process will stay in the "insession" status until it is manually deleted or manually changed to the "new" status. The command 'ispman.commitSession' can be used to change the status to new.

ISPMan-agent

Once you have fully installed ISPMan, your servers will each run the ISPMan-agent. Every few minutes, ISPMan-agent checks your LDAP directory for pending processes. The agent uses the machine's hostname as an LDAP filter, and executes any processes with a status of “new”. As processes are completed, they are deleted from the LDAP directory.

If the agent is unable to complete a task, the agent may exit. For this reason, it is highly recommended that ISPMan-agent be run as a “watched” process. The supervise program in the daemontools package is commonly used. http://cr.yp.to/daemontools/supervise.html

Chapter 2:Installing ISPMan

Quickstart

Overview and pre-installation checklist

Overview of the process:

A) Download, configure and install ISPMan then populate LDAP.

B) Log into the ISPMan web interface and finish configuration

C) Setup ISPMan-agent with a startup script

D) Configure daemons/services to use LDAP

Before You Begin

• Choose an LDAP password. This will be needed when you run “make ispman” and when you first login to your ISPMan management website.

• Choose a password for the Cyrus mail system. You'll need this when you run “make ispman” and when you test Cyrus using cyradm.

• If you don't have LDAP installed, install it. Don't worry about the configuration, since ISPMan will create an example config file for you to copy over later.

# apt-get install libdigest-md5-perl slapd

Configuring and Compiling ISPMAN

1. Create a user called ispman and assign ownership of the /opt directory. If you have other stuff in the /opt directory, make your permissions more granular.

# adduser –system ispman
# chown ispman /opt -R

2. Login as ispman and download the latest ispman.tar.gz to /opt. Check the home page and install the current version. (I'll disclose that we buy bandwidth from Time Warner Telecom, so I use their mirrors for speed).

# su – ispman
$ wget http://twtelecom.dl.sourceforge.net/sourceforge/ispman/ispman-1.1.tar.gz
$ tar zxf ispman-1.1.tar.gz 

3. Go into the unpacked ispman-1.1 direcory and do a cvs update. This applies the latest bug fixes and patches.

$ cd ispman-1.1
$ apt-get install cvs
$ cvs -z3 update -dP

4. Configure and make ISPMan. If you want to install to a directory other than /opt/ispman, use the –prefix option in the ./configure script.

$ ./configure
$ make

5. To run the web administration interface on this machine, do this:

$ make ispman install_ispman_common install_ispman_web

6. For a new installation ONLY, initialize LDAP. (You'll need root access)

# cp /opt/ispman-someversion/tmp/conf/slapd.conf /etc/ldap/slapd.conf 
# cp /opt/ispman-someversion/tmp/conf/schema/* /etc/ldap/schema/ 
# /etc/init.d/slapd restart 
# make ldif-install

7. For an existing installation, do not initialize LDAP. Check the release notes for any special upgrade instructions. You may need to dump an LDIF, run a schema fix script and then reimport the LDIF..Also, be sure to copy the latest schema files over.

NOTE: (ldif export, patch and import steps not shown)

# cp /opt/ispman-someversion/tmp/conf/schema/* /etc/ldap/schema/ 
# /etc/init.d/slapd restart 

Setting up ISPMan Web Interfaces

This will help you get the ISPMan web interface running along with the Customer Control Panel.1. Install apache-ssl

# apt-get install apache-ssl

2. Modify /etc/apache-ssl/httpd.conf, customizing it for your domain. I prefer to access ISPMan by typing “https://ispman.yourdomain.tld”. I also like to restrict access only to certain IPs.

<VirtualHost *>
        SSLEnable
        ServerName      ispman.yourdomain.tld
        DocumentRoot    /opt/ispman/htdocs
        Options    FollowSymlinks ExecCGI
        <Directory  /opt/ispman/htdocs>
                Options ExecCGI
                AddHandler cgi-script .cgi
                DirectoryIndex index.html index.cgi
                AllowOverride None
        </Directory>
</VirtualHost>

3. Restart apache-ssl

# /etc/init.d/apache-ssl restart

4. Modify your local hosts file or existing DNS to access the server. At ispman.yourdomain.tld. You should now be able to access ISPMAN at

https://ispman.yourdomain.tld/

If unable to access the website, check /var/log/apache_ssl/error_log and fix any wrong lines and try again.

5. From your ISPMan Management Page, click Admin, and login as an Administrator. The default username is ispman, the password is the LDAP password.

Web Global Configuration

ISPMan needs to be configured with your network environment before you can add domains. Here is a brief overview of each screen and how I setup my installation:

Apache Config

 Apache stop command - /etc/init.d/apache stop
 Webalizer command - /usr/bin/webalizer
 Apache vhost file - /etc/apache/vhosts.conf
 Apache start command - /etc/init.d/apache start
 Apache graceful reload command - /etc/init.d/apache reload
 Default vhost IP -- * (Allows it to work on any machine)
 Apache reload command - /etc/init.d/apache reload

NOTE: Directories, paths and commands may vary, please verify your are working. The vhosts file is very important, since it will be included in your httpd.conf file later.

Mail Configuration

 IMAP server - cyrus
 Server Maps command - /usr/sbin/postmap
 SMTP reload command - /etc/init.d/postfix reload
 Map files directory - /etc/postfix
 Mailbox Prefix - user.

DNS Settings

 Named Start Command - /etc/init.d/bind start
 Primary DNS (name) - ns1.yourdomain.tld
 ISPMan's DNS data directory -  ispman/
 Primary Mail Server (name) - mail.yourdomain.tld
 Named data directory - /var/named  
 Secondary Mail Server (IP) - xxx.xxx.xxx
 DNS Backend - bind8
 Secondary DNS (IP) - xxx.xxx.xxx.xxx
 Reverse Zone directory - rev/
 Header file for named.conf - /opt/ispman/templates/named.conf.header 
 Named Reload Command - /etc/init.d/bind reload 
 Footer file for named.conf -/opt/ispman/templates/named.conf.footer 
 Named Stop Command - /etc/init.d/bind stop 
 Secondary Zone directory - sec/ 
 Named Config File  -/etc/named.ispman.conf
 Primary DNS (IP) - xxx.xxx.xxx.xxx
 Secondary Mail Server (name) - mail2.yourdomain.tld
 Secondary DNS (name) - ns2.yourdomain.tld
 Primary Mail Server (IP) - xxx.xxx.xxx.xxx
 Catch all CNAME  - none
 Primary Zone directory - pri/
 Hostmaster's email address - hostmaster.yourdomain.tld
 Named User - root 

Add Admins, Resellers and Clients

Admins are typically people who work within your organization. They can add and view settings for all domains within ISPMan.

1. To add an Administrator, you must be logged in with an account that has authority level 100. For most installations, just login as ispman with the LDAP secret password you entered while installing ISPMan.

2. Click Configure > Administrator accounts. Then click “Add a new admin”

3. This chart should help you choose the right authority level for your new administrator.

NOTE: Administrator Authority Levels

Assign a level of 20 for Help Desk Staff
0 to 69 - Add Resellers, Clients, Domains, Mailboxes
70 to 99 - The above, plus delete domains
100 - The above, Add administrators, manage configuration

Before adding any Clients, Add at least 1 Reseller. For even the smallest installation, create a Reseller to represent your origanization.

To add a Reseller, Click on the Resellers button. You must be logged in as an Administrator. Then click “Add a new Reseller”. Choose the maximum number of clients the Reseller can add.

To add Clients to ISPMan, click in the Clients button. Administrators AND Resellers can add client.

Adding Hosts and Hostgroups

ISPMan depends on hosts and hostgroups to setup new email accounts, domains, and virtualhosts. Adding a domain will create processes for all hosts in the DNS hostgroup. Making a new virtualhost will make processes for the machine that will host the domain. Similarly, adding a new user will create processes for the chosen mailhost and fileserver.Make sure to install ispman-agent on each host.

Setting up ispman-agent

Make sure each machine assigned to a hostgroup has ispman-agent installed.

1. It is recommended that ISPMan-agent be run as a supervised process. Most administrators install daemontools, make a subdirectory for ispman.

# mkdir /supervise/ispman

2. Create a /supervise/ispman/run script like so:

#!/bin/sh
echo Starting ispman agent
exec /opt/ispman/bin/ispman-agent nodetach

3. Go ahead and start ispman-agent.

# chmod +x /service/ispman/run
# svc -u /service/ispman

4. Next, check the logs in /var/log/ispman/ to see whats happening.

The biggest problem with agents not completing tasks is a non-matching hostname. If you are new to supervise, here's a quick guide to controlling processes. I also have hacked up an init.d script, see the mailing list archives.

svc -h /service/yourdaemon: sends HUP
svc -t /service/yourdaemon: sends TERM and restarts
svc -d /service/yourdaemon: sends TERM and leaves it down 
svc -o /service/yourdaemon: runs the service once 

5. You will want to make sure that ispman-agent starts on bootup.

Securing and Testing LDAP

This provides basic security for LDAP using tcpwrappers. You can supplement this with LDAP ACLs and iptable rules.

1. You should limit access to your LDAP directory atleast to the IP level. Edit /etc/hosts.allow and add IP addresses of the servers that need remote LDAP access.

slapd: XXX.XXX.XXX.XXX 127.0.0.1 : ALLOW

NOTE: YOU MUST HAVE A NEWLINE AFTER YOUR ENTRIES OR THEY WON'T WORK.

2. Deny all other machines access by making /etc/hosts.deny look like so:

slapd: ALL : DENY

3. Verify this works by using ldapsearch locally and from a remote machine not in the allowed file. (requires ldap-utils package)

$ ldapsearch -xvu -LLL -b "o=ispman" ispmanVar=*

Setting up PAM_LDAP

This basically allows any daemon using PAM for authentication to use LDAP. Some programs like Cyrus don't authenticate directly from LDAP without patches. This lets them 'ask' PAM for authentication. PAM then queries LDAP and returns the answer in an understandable way.

1. Install PAM_LDAP. The default options are fine, we'll edit the config file directly.

# apt-get install libpam-ldap

2. Setup /etc/pam_ldap.conf something like this:

  host 127.0.0.1
  base o=ispman 
  ldap_version 3

Requirements

If you want to test ISPMan, you'll need at least 1 machine. You can find packages for the applications (like Postfix, Cyrus, Bind, PureFTPd) in most distributions. Detailed steps for configuring applications are in chapter 3.

As you plan your installation, determine what external applications you will use.

Chapter 3: Configuring external apps for ISPMan

Compatibility Matrix

This compatibility matrix gives a brief overview of external applications and how well you can expect support within ISPMan. Understand that ISPMan is open source, and completely modifiable. As with so many things, if you have the determination, need, and/or money, you can “make” things supported.

Bind 8

1. Install Bind8

# apt-get install bind

2. Add this line to your bind configuration file (mine is /etc/named/named.conf)

include "/etc/named.ispman.conf";

NOTE: The ; at the end is critical, without it the logs will say:

/etc/bind/named.conf:67: parse error near 

3. Make sure bind is looking in the right place for the zone files. I ended up linking /etc/bind with /var/named, since ISPMan was mad when I tried using /etc/bind during my original install. This should no longer be an issue, but this is what I had to do.

# ln -s /etc/bind/ /var/named

4. Reload bind and check the logs, it should be happy.

# /etc/init.d/bind reload 

NOTE: You can force DNS updates by running ispman.ldap2named

Bind 9

1. Install Bind9

apt-get install bind9

2. Add this line to your bind configuration file (mine is /etc/named/named.conf)

include "/etc/named.ispman.conf";

NOTE: The ; at the end is critical, without it the logs will say:

/etc/bind/named.conf:67: parse error near 

3. Make sure bind is looking in the right place for the zone files. Your web configuration options for DNS need to indicate you are using bind9.

4. Reload bind and check the logs, it should be happy.

# /etc/init.d/bind9 reload 

NOTE: You can force DNS updates by running ispman.ldap2named

Bind 9-SDB

This works well except that wildcards do not work. It is undocumented, but should be straightforward.

TinyDNS/djbdns

This can work, but the setup steps are undocumented.

Postfix 1

1. Install Postfix with LDAP support. You will probably also want TLS and SMTP Authentication support. It is quite easy to get PLAIN and LOGIN SMTP Auth working, however, you should use TLS/SSL with these mechanisms since they are cleartext.

# apt-get install postfix-ldap postfix-tls libsasl-modules-plain

2. Setup PAM for SMTP Authentication. Edit /etc/pam.d/smtpd like so:

auth    sufficient      pam_ldap.so 
account sufficient      pam_ldap.so

3. Setup the Postfix to SASL for SMTP Authentication. This just makes the directory for the next step.

# mkdir /etc/postfix/sasl 

4. Create /etc/postfix/sasl/smtpd.conf like so:

pwcheck_method: pam

5. Edit /etc/postfix/master.cf

Make sure smtpd is NOT chrooted -- This lets it do SMTP Auth.

WARNING: SMTP Authentication is an addon to postfix, it does decrease the overall security. You have been warned. Non-chrooting is considered evil by many. You really should make this run in a chroot environment.

6. Create TLS cert and key in /etc/postfix

# openssl req -new -x509 -nodes -out cert.pem 
# chown root.postfix *.pem
# chmod 640 *.pem  

7. edit /etc/postfix/main.cf using the example in build/install-data/examples/postfix_configuration/main.cf.reasonable with these exceptions:

mailbox_transport=cyrus
# NOTE: myorigin MUST match the maildrop hostname in ISPMan.
myorigin = /etc/mailname

#I didn't need these features, so I commented them out.
#relocated_maps = hash:/etc/postfix/relocated
#smtpd_sender_restrictions = hash:/etc/postfix/access

# SMTP Auth Stuff see http://killyridols.net/postfixtlssmtpauth.shtml
smtpd_sasl_auth_enable = yes 
smtpd_sasl_security_options = noanonymous
smtpd_sasl_local_domain = $myhostname
smtpd_recipient_restrictions = permit_sasl_authenticated,  check_relay_domains

#For STARTTLS SMTP
smtpd_tls_cert_file = /etc/postfix/cert.pem
smtpd_tls_key_file = /etc/postfix/privkey.pem
smtpd_use_tls = yes
tls_random_source = dev:/dev/urandom
tls_daemon_random_source = dev:/dev/urand

8. Test it using telnet.

$telnet localhost 25

You should see something like:

Connected to 10.98.76.54
Escape character is '^]'.
220 test.somewhere.com ESMTP Postfix

If postfix drops the connection, read syslog. Most of my install problems are with SMTP auth settings, try commenting that out and restarting if postfix components are dying. Also ensure that your pam files are correct and that smtpd is NOT in a chroot environment.

LMTP can speed up stuff, you can set that up if you like. In fact, once you get that running, you really SHOULD set that up.

Postfix 2

Here's an example using Postfix2 with Cyrus delivery using LMTP delivery adapted from configs posted at http://www.nyetwork.org/~wim/example-mail-configs-wim.tar.gz by Wim Kerkhoff. Thank you!

1. Install postfix2, make sure you have LDAP and SMTP Authentication support.

2. Setup your main.cf in a similar fashion, customize the LDAP settings. Note that this also supports POP-before-SMTP.

command_directory = /usr/sbin
daemon_directory = /usr/lib/postfix
program_directory = /usr/lib/postfix

smtpd_banner = $myhostname ESMTP $mail_name
setgid_group = postdrop
biff = no

myhostname = master.hoster123.net
mydomain = hoster123.net
alias_maps = hash:/etc/aliases
alias_database = hash:/etc/aliases
myorigin = $myhostname
mydestination = $mydomain, $myhostname, localhost.localdomain, localhost, hash:/etc/postfix/destination_domains
virtual_maps = hash:/etc/postfix/virtual, ldap:ldapvirtual
mailbox_transport = lmtp:localhost:2003
relayhost = 

# add the IPs/32 of other servers in the cluster to the next line, or the whole /24 that they are in
mynetworks = 127.0.0.0/8 
mailbox_size_limit = 0
recipient_delimiter = +

default_transport = smtp
inet_interfaces = all

ldapvirtual_server_host = localhost
ldapvirtual_server_port = 389
ldapvirtual_bind = no
ldapvirtual_bind_dn = cn=Directory Administrator, o=ispman
ldapvirtual_bind_pw = pass_here
ldapvirtual_timeout = 5
ldapvirtual_search_base = o=ispman
ldapvirtual_query_filter = (|(mailLocalAddress=%s)(mailAlias=%s))
ldapvirtual_result_attribute = mailRoutingAddress,mailForwardingAddress
ldapvirtual_lookup_wildcards = no

relocated_maps = hash:/etc/postfix/relocated
smtpd_sender_restrictions = hash:/etc/postfix/access
transport_maps = hash:/etc/postfix/transport

local_destination_concurrency_limit = 2
default_destination_concurrency_limit = 10

debug_peer_level = 3


# for authentication
smtpd_sasl_auth_enable = yes
smtpd_sasl_security_options = noanonymous

# can use this next line instead, when testing CRAM-MD5
# smtpd_sasl_security_options = noanonymous, noplaintext

smtpd_sasl_local_domain =
broken_sasl_auth_clients = yes
smtpd_sender_restrictions = 
	check_client_access hash:/var/lib/pop-before-smtp/hosts,
	permit_sasl_authenticated,
	permit_mynetworks,
	reject_unauth_destination
smtpd_recipient_restrictions =
	check_client_access hash:/var/lib/pop-before-smtp/hosts,
	permit_sasl_authenticated,
	permit_mynetworks,
	reject_unauth_destination

3. Setup PAM for SMTP Authentication. Note that the filename used differs than from Postfix 1.1's PAM example. Edit /etc/pam.d/smtp like so:

auth required /lib/security/pam_ldap.so 
account required /lib/security/pam_ldap.so 
password required /lib/security/pam_ldap.so use_authtok 
session required /lib/security/pam_ldap.so 

3. Make sure your main.cf is sane. The defaults should work.

4. Ensure cyrus is setup to accept LMTP on port 2003. If they are on the same machine editing cyrus.conf's like so should work fine. Note that you should not allow just any machine on the Internet to access your LMTP.

lmtp	cmd="lmtpd -a" listen="localhost:555" prefork=1 maxchild=20

3. Restart postfix2 and cyrus to make your changes active.

# /etc/init.d/postfix restart
# /etc/init.d/cyrus restart

3. Test it using telnet.

$ telnet localhost 25

You should see something like:

Connected to 10.98.76.54
Escape character is '^]'.
220 test.somewhere.com ESMTP Postfix

If postfix does not respond right away check your logs. Verify that you have POP before SMTP setup, or have commented out that section of the config.

Cyrus 2.1

1. Install Cyrus 2.1. Make sure you get the correct SASL modules installed, remove any you won't use since this often confuses mail clients.

On Debian:

Add these lines to /etc/apt/sources.list since cyrus2.1 is a backport.

  #Cyrus21 Backports
  deb http://people.debian.org/~hmh/woody/ hmh/cyrus/
  deb http://people.debian.org/~hmh/woody/ hmh/misc/

Then update and install Cyrus2.1

# apt-get update 
# apt-get install cyrus21-pop3d cyrus21-common cyrus21-imapd \
cyrus21-admin cyrus21-admin cyrus21-clients cyrus21-doc gawk \
libcyrus-imap-perl21 libsasl2 libsnmp-base libsnmp4.2 \
sasl2-bin libsasl2-modules libsasl2 sasl2-bin

NOTE: You could use just libsasl2-modules-plain instead of libsasl2-modules but it had dependency problems when I tried it.

2. Setup /etc/imapd.conf. This setups an admin, uses SASL for password authentication and sets us up to support SSL/TLS. Never read mail using an admin account.

admins: cyrus
popminpoll: 0
allowplaintext: yes
sasl_minimum_layer: 0
sasl_mech_list: plain login
sasl_pwcheck_method: saslauthd
lmtp_downcase_rcpt: yes
tls_ca_file: /etc/ssl/cyrus.pem 
tls_cert_file: /etc/ssl/cyrus.pem 
tls_key_file: /etc/ssl/cyrus.pem

4. Edit /etc/cyrus.conf and make sure IMAP and POP3 are listening.

imap      cmd="imapd -s -U 30" listen="imap" prefork=0 maxchild=100
pop3      cmd="pop3d -U 30" listen="pop3" prefork=0 maxchild=50

5. If you want TLS support, create SSL key and pem file for cyrus

# openssl req -new -nodes -out req.pem -keyout key.pem  
# openssl rsa -in key.pem -out new.key.pem
# openssl x509 -in req.pem -out ca-cert -req \
-signkey new.key.pem -days 999 
# mv new.key.pem /etc/ssl/cyrus.pem
# cat ca-cert >> /etc/ssl/cyrus.pem
# chown cyrus.mail /etc/ssl/cyrus.pem
# chmod 600 /etc/ssl/cyrus.pem 

6. Setup PAM files for Cyrus. This lets the authentication go Cyrus to SASLAUTHD to PAM_LDAP. The files are /etc/pam.d/imap and /etc/pam.d/pop.

auth    sufficient      pam_ldap.so 
account sufficient      pam_ldap.so

7. Add cyrus to the sasl group

# adduser cyrus sasl
 

8. Make sure saslauthd uses PAM. I modified /etc/default/saslauthd.

start=yes
MECHANISMS="pam"

NOTE: I've seen configs where saslauthd goes straight to LDAP.

The example I followed didn't work, so I stuck with pam. YMMV.

9. Restart saslauthd and cyrus

# /etc/init.d/cyrus21 restart
# /etc/init.d/saslauthd restart

10. Test the setup by doing this:

# cyradm --user cyrus localhost
   localhost password:  
   localhost> lm   

If you don't get a hostname> prompt, check the syslog for errors. Verify LDAP is running. If you have other SASL mechnanisms installed, you may need to use “cyradm --auth plain –user cyrus localhost”.

THEN the ultimate: Create a domain with an email user and verify! Commit the session and verify that the Process list is empty before continuing. If processes stay in the queue, verify the machine's hostname and make sure you have at least 1 host in the mailserver group.

$ telnet localhost 110
   USER someuser_domain_com
   PASS password
   LIST 

Courier

No one has contributed an example. Googling on “courier postfix ldap” yields a few howtos. It has been done, I just don't have a clean easy set of install steps to show you. If you decide to choose Courier, realize that Cyrus was the original IMAP/POP daemon “target” and the best supported since most of the developers use Cyrus.

Perdition

Perdition allows your users to use user@domain.com or user_domain_com as user names for POP, and IMAP to Cyrus or Courier. This is needed in Cyrus since '@' and '.' are not allowed in account names, so ISPMan uses '_'. Larger ISP's will have Perdition on separate server, as a IMAP/POP multiplexer. Smaller ISPMan installations can still benefit from Perdition by installing it on the same machine. Perdition can also be used to help migrate existing email user by adding or stripping off parts of the user name. However, that is outside the scope of what we're doing.

Even if you only have a single email server, learning Perdition can help your system grow. Don't think of Perdition only in terms of username translation. Think of it as a in terms of easy and planned scalability. Configuring email programs to go to Perdition makes your life easier in the long run. With Perdition “directing” traffic, splitting the load between several email servers is much easier. Adding new servers requires NO changes to what names customers must “plug in” to their email program.

1. If running ISPMan 1.0 or higher skip to step 2. If you are running 0.9.7 or lower, you should upgrade or read the archives on patching the older version. Realize that if you hit a bug in an old version, the answer will probably be, upgrade to the latest released version.

2. Make sure you have access to your LDAP server(s) from the machines that will run Perdition.

You may have to edit tcpwrappers (/etc/hosts.allow), iptables/ipchains or other firewall rules depending on your security setup.

3. Verify the LDAP server is reachable, something like this works well

$ ldapsearch -xvu -h 127.0.0.1 -LLL -b "o=ispman" ispmanVar=*

4. Install perdition

You can use packages for your distribution, or compile from source available from: http://www.vergenet.net/linux/perdition/download/

# apt-get install perdition perdition-ldap

5. Configure Perdition

Keep in mind that your map_library file may be in a different location. When you “go live” you should have multiple LDAP servers. As long as they are all on standard ports, set these up on the map_library_opt line.

A basic /etc/perdition/perdition.conf looks like so:

map_library /usr/lib/libperditiondb_ldap.so.0
map_library_opt "ldap://127.0.0.1/o=ispman?mailroutingaddress?sub?(uid=%25s)"
server_ok_line
username_from_database

A single domain setup config file will probably look more like

map_library /usr/lib/libperditiondb_ldap.so.0
# The following map_library_opt line needs to be on a single line
# Customize the @azxws.com to match your domain.
map_library_opt “ldap://127.0.0.1//o=ispman?mailroutingaddress?sub?(uid=%25s@azxws.com)"
server_ok_line
username_from_database

A multi-LDAP server config file will probably look more like :

map_library /usr/lib/libperditiondb_ldap.so.0
# The following map_library_opt line needs to be on a single line
map_library_opt “ldap://127.0.0.1 192.168.2.10 192.168.3.10
/o=ispman?mailroutingaddress?sub?(uid=%25s)"
server_ok_line
username_from_database

6. Make sure Perdition starts on boot up and start it manually now.

# /etc/init.d/perdition start

7. Verify the account and password are correct by logging in directly

against the mail server. I usually do something like this against my Cyrus mail server:

$ telnet localhost 110
  USER user_domain_com
  PASS thepassword

Then I repeat these steps using the Perdition proxy. If it fails, check the logs (perdition uses syslog). You may also use a system debugger (like strace) and a network sniffer to verify LDAP is functioning. I have also had good success setting an outgoing_server in perdition.conf as a debugging measure

Once you have underscore names working, use the user@domain.com name through Perdition.

Apache 1.3

1. Install apache

# apt-get install apache

2. Setup apache for ISPMan hosting by adding a section like this to end of httpd.conf :

UseCanonicalName Off  
NameVirtualHost * 
#this is where ispman-agent adds vhosts, must match ISPMan web config
Include /etc/apache/vhosts.conf

NOTE: With UseCanonicalName Off the server name comes from the contents of the Host: header in the request.

3. Restart apache and make sure the logs are clean.

# /etc/init.d/apache restart

NOTE: You can force an update by running ispman.ldap2apache. I have run into bugs where a vhost would still not update and so I erased the vhosts.conf.hash and then ran ispman.ldap2apache

Apache 2

This can work, however it is undocumented. I am unsure of the config file changes needed. You will most likely need to edit the vhost.template file to have proper configuration options for apache 2. Subscribe to the mailing lists and search the arcives for success stories.

ProFTPd

ProFTPd does not currently support space quotas in the same format as ISPMan. I have used PureFTPd, though and it works. Point it at your LDAP server(s) with the right Base DN (usually o=ispman).

PureFTPd

PureFTPD 1.2 is also reported to work with some changes. However, I have not tested this.

1. Install pureftpd 1.x

Debian Notes: I searched www.apt-get.org for a backport.

In /etc/apt/sources.list:

#PureFTPd
deb http://pureftpd.sourceforge.net/debian/woody/ .

Then I updated and installed:
# apt-get update
# apt-get install pure-ftpd-ldap 

2. Configure /etc/pure-ftpd.conf to enable LDAP. I changed these 2 lines. Yes, you could also run this through PAM_LDAP.

LDAPConfigFile                          /etc/pureftp-ldap.conf
#PAMAuthentication             yes

3. Setup the LDAP config file, in this case /etc/pureftpd-ldap.conf . In the past I had used a uid filter, pureftpd complained when I did that with this release, so my config is simpler.

LDAPServer localhost
LDAPPort   389
LDAPBaseDN o=ispman

4. Create a domain and make sure FTP access is enabled.

5. Restart pureftpd and test:

# /etc/init.d/pureftpd restart

$ ftp localhost 
   USER www.domain.com   
   PASS password      

RADIUS

This is undocumented. Please submit something if you have made a RADIUS server work with ISPMan. You should be able to get something going by using the LDAP Implementation Howto. Time has not permitted me to set this up and test.

Chapter 4: What you need to know about LDAP

Gentle Introduction to LDAP

An LDAP server provides directory-like information to LDAP clients. Since LDAP is an open standard, cross platform access is straighforward. Early LDAP directories were basically company addressbooks. ISPMan uses LDAP to store network and host configuration information.

Beside being open and lightweight, LDAP is also

1. easy to replicate and make your LDAP system distributed. Most all clients can support multiple LDAP servers and failover automatically.

2. easy to use the information in LDAP for other purposes. With cool “bridges” like PAM_LDAP, you can setup SAMBA to PAM to LDAP in a snap. You can also centralize user management and create company directories at the same time.

3. Very granular in allowing access.

The format of information in the LDAP directory is determined by the schema. The schema prevents things from just getting “thrown in” to LDAP.

LDAP connections can be anonymous, or authenticated. When using authentication, you will need a Bind Distinguished Name (Bind DN), and a password. LDAP is a cleartext protocol running on TCP port 389. LDAP can be encrypted by using TLS/SSL, often called LDAPS. LDAPS runs on TCP port 636. You can use LDAPS by compiling against OpenSSL or by wrapping it stunnel. Note that using L requires an LDAPv3 server, since the standard requires that a normal LDAP connection can by switched on demand from plaintext to a TLS connection.

Replication

??Show diagram using slurpd to several slave slapds.

ISPMan LDAP Layout

The LDAP layout is underneath your root DN, typically o=ispman.

ou=admins

Admins, plus cyrus and ispman accounts.

ou=conf

Default configuration values for apache, dns, mail, system. Basically what you see when you click Configure inside the Admin interface.

ou=counter

This tracks the next userid, procesid, resellerid,clientid, aclidcounter.

ou=domainServices

ou=DNS

ou=hosts

ou=hostgroups

ou=ispman

Resellers and Clients login names and passwords are stored here.Each domain managed may contain the ispmanClientID. This grants access to the client and reseller.

ou=processes

Configuration changes create processes for each host here. Each machine running ispman-agent uses the local hostname as a filter to discover its' work items. ispman-agent deletes successfully completed items.

ispmanDomain=example.com

Each domain managed in ISPMan gets a ispmanDomain branch.

ispmanDomain=example.com,ou=httpdata

ispmanDomain=example.com,ou=httpusers

This contains web users. These users differ from those in ou=users since they have no mailbox.

ispmanDomain=example.com,ou=ispmanMailGroup

This subbranch contains all mail group information. Sending an email to the mailgroup name results in emails to all members.

ispmanDomain=example.com,ou=users

Contains “full” users of the domain. Each user gets a mailbox and can receive mail.

Filter Examples

Needs to be extended.

Uid lookup

Display information about a uid by using a filter like this:

$ ldapsearch -xv -LLL -b "o=ispman" '(uid=user@example.com)'

Process Lookup

Ispman-agent does a process lookup using the machine hostname. The lookup looks like this: (where hostname is the actual machine's hostname)

$ ldapsearch -xv -LLL -b "o=ispman" '(objectclass=ispmanProcesses)(ispmanStatus=new)(ispmanHostName=host.domain.com)'

You can get an overview of the processes by using this filter:

$ ldapsearch -xv -LLL -b "o=ispman" '(objectclass=ispmanProcesses)'

Chapter 5: Day to Day Domain Management

Domains

Note to Resellers and Administrators: Verify that the customer has a client login so they can manage their domain. Be sure they are assigned to your reseller account.

Note to Client: If logging in as a client, you may be able to create your own domains. Your Reseller or ISP has an option to allow you to create up to a maximum number of domains. If you can't add a domain, ask your Reseller for assistance.

1. Click on Add Domain, filling out these pieces of info:

Understanding Domain Types within ISPMan

There are 4 types of domains within ISPMan. The different types of control how DNS is managed.

primary - DNS is managed by ISPMan (default)

slave/secondary - DNS is managed somewhere else and we are a “slave” DNS server

nodns - ISPMan should not worry about DNS or DNS is hosted elsewhere

replica - The domain is a copy of another ISPMan domain

Domain Name = customer.com

Domain Owner = Who pays for this

Domain Customer = Who owns the domain

Select the Domain Client from the drop-down menu

Choose the services needed for this domain -- Web, Mail or both.

Then click Next

TIP: Imagine you just registered customer.org and wanted email and web pages to mirror what is already setup for customer.com. As long as customer.com is a primary domain, you can make this happen by making customer.or a replica of customer.com

2. The next screen looks like this:

Home Directory = Don't touch this unless you have a good reason

FTP Password = allows domain wide FTP access and access to the customer

control panel.

Max Email accounts = Leave blank, unless this account has a limit.

Max Virtual Hosts = Leave blank, unless this account has a limit.

Max Storage Space = see note below

MAX STORAGE SPACE EXPLAINED

Max Storage Space refers ONLY to FTP space for the domain. Email quotas are per user. Right now, no FTP server correlates the Max Storage Space to the users and vhosts in the domain. Once the /ispman/domains/domain.com with all subdirectories is over the limit, only the domain level account (in the form of example.com) is locked out. Users and vhosts who haven't hit their quotas can keep uploading. You should focus on setting reasonable quotas for users and vhosts under the domain.

Mail Delivery = MX


NOTE: If email is NOT handled by ISPMan for this domain, be sure the DNS MX records have the preferred email server(s) listed.

Click Save Entry

3. Setup DNS extras:

Under A:

alias @ to the same IP as www

(useful for making customer.com go to www.customer.com)

4. Be sure to click "Commit session" when done.

Users (domain, web, dialup, wireless)

To add an email account Once t

in ISPMan follow these steps:

1. Select the domain to manage.

2. Click Add user under the Manage users section.

3. Fill out the information about the new user. Username can contains characters a - z. No accent characters, numbers or punctuation marks are allowed. A good username is first character of firstname plus the lastname

• Username - user name

• Surname - last name of the user

• FirstName -first name of the user

• Password - user password

NOTE: Depending on your configuration, the password field may appear scrambled the next time you view the user. This is normal.

• Create mailbox on server - choose a server from the mailstore group to

• create a mailbox, or None (no mailbox, just forwarding)

• FTP Access - enable or disable a private ftp folder

• Max FTP Storage Space (in MB) - set an ftp storage quota

• Max Mail Storage Space (in MB) - set an email storage quota

• Set the Password field to an agreed upon password with the user or just assign a password and inform the user.

• Click the "Save Entry" button.

• Be sure to click "Commit Session" to activate your change.

  Catchall Email Accounts

  To create a "catchall" account, edit the user and add @domain.com to their aliases.

Uploading websites

Most sites using ISPMan will allow FTP access only. These steps are suitable for sending helping customers to upload their sites.

NOTE: With some workarounds, it is possible to allow sftp using ssh.

FTP server The hostname of the virtualhost. If you were editing www.customer.com, the FTP server would be www.customer.com.

username The full name of the virtualhost. If you were editing www.customer.com, the username would be www.customer.com.

password Whatever you set above. The website files should be uploaded to the htdocs folder.

Website Options/ACLs

Adding a virtualhost is straightforward.

1. Select the domain to manage

2. Under "Manage virtualhosts" click Add virtualhost.

3. A JavaScript box pops up, type "www" two times with no quotes.

4. Set virtualhost aliases to these 2 lines (changing the domain name) customer.com *.customer.com.

   These aliases tell the webserver to always serve the website files at: /ispman/domains/customer.com/vhosts/www/htdocs NOTE: If you have multiple virtualhosts/websites, don't use *.customer.com as an apache alias, use simply www.customer.com. Using *.customer.com may block access to the second virtual host.

5. Make sure "FTP Access" is enabled, and set an "FTP password".

6. Click the "Update" button.

7. Be sure to click "Commit session" when done.

ADVANCED TIP: To add a second virtualhost, like private.customer.com, use the above steps, but at step 3, type private twice. Your ftp server would be private.customer.com and FTP username would be private.customer.com.

Chapter 6: Migrating to ISPMan

Migrating is always a stressful time. If you choose to migrate, planning and verification are very important. ISPMan comes with lots of command line tools that can be scripted. Please note that http:/www.ispman.com lists contact information if you'd like to pay someone else to do your migration.

Command line tools

See the ISPMan CLI documentation for specific example of adding domains, DNS records, and users.

DNS

Most domains can simply be added using the ispman.addDomain script. Entire zone files can be added using ispman.zoneparse.

Users

When ISPMan was being built, the internal username looked like user_domain_com. This was due to a limitation of the Cyrus mail backend. As of the 1.0 release, usernames can be referenced as user_example_com OR user@example.com. Cyrus still does not like user@example.com, but using an IMAP/POP proxy like Perdition works around this problem.

Some sites have a single large domain where the majority of users simply use usernames like 'twasson'. This can easily result in name collisions as the userbase grows. It is possible to continue authentication users with short usernames with the Perdition IMAP/POP proxy. An example filter for this would be:

“ldap://127.0.0.1//o=ispman?mailroutingaddress?sub?(uid=%25s@azxws.com)"

Another idea for dealing with a username migration is to add another userid. This makes 3 uids for each user: username, username_example_com, and username@example.com. This is also known as the “3 UID hack”.

Email

Almost every installation will need to do some sort of email migration. Fortunately, it is possible to run multiple servers and route users to the correct mailbox using Perdition. (I really can't say enough good things about Perdition.)

If the old server supports subfolders, you can be an evil sysadmin and make users move their own junk, or use some tools to move it for them. 2 tools that can be used for this are mailutil, or imapsync. Here's an example of imapsync moving a mailbox from 1 server to another:

/usr/bin/imapsync --host1 old.email.server --user1 wasson --password1
MyPassword --host2 new.email.server --user2 wasson_azxws_com –password2 MyPassword

Websites

The main considerations for moving large amounts of websites are reliance on scripts, understanding any reliance on bankend databases (like MySQL or PostgreSQL).

Chapter 7: Appendix

Needs to be written.

A. Glossary

Replica domain

Slave Domain

Secondary Domain

Primary Domain

nodns Domain