1. Introduction
PPPoE stands for Point to Point Protocol over Ethernet. It is an OSI Layer Two protocol which uses the PPP (Point to Point) protocol to connect a client system to a server system in a one to one network link. All traffic for a PPPoE connected client must go thru the PPPoE server to reach the client. A PPPoE server can therefore be used to route, NAT, firewall, and perform QOS traffic shaping. Since the newer versions of the PPP daemon support RADIUS attributes client authentication and configuration can be done remotely at the RADIUS server. Linux is an excellent, high performance operating system which can easily support a large number of PPPoE clients upon modest hardware platforms.
This document will probably be of interest to Internet Service Providers, Information Technologists, and other Network Managers.
1.1 Disclaimer
No liability for the contents of this documents can be accepted. Use the concepts, examples and other content at your own risk. As this is a new edition of this document, there may be errors and inaccuracies, that may of course be damaging to your system. Proceed with caution, and although this is highly unlikely, the author(s) do not take any responsibility for that.
All copyrights are held by their respective owners, unless specifically noted otherwise. Use of a term in this document should not be regarded as affecting the validity of any trademark or service mark.
Naming of particular products or brands should not be seen as endorsements.
You are strongly recommended to make a backup of your system before major installation and backups at regular intervals.
2. Hardware
Linux is a high performance OS. As evidenced by the current trend to deliver Linux firmware on SOHO routers and wireless devices. PPPoE is a CPU intensive task and running multiple instances of a server daemon will inevitably eat up memory. We have built a number of these PPPoE server systems over the past couple of years and have observed what appears to be a linear relationship between the number of simultaneously connected clients and system CPU and memory requirements.
It is safe to estimate that one connected client will require about 2 Mhz of system CPU on an INTEL Pentium-III processor. It is equally safe to estimate that one connected client will require about 2 Mbytes of system memory. These numbers have been estimated based upon our experience with previous PPPoE server systems we have built that handled small numbers of connections (terminations). The most simultaneous terminations we have observed on one of our systems is 128. That observation was made on a 266 Mhz Pentium with 256 Mbytes of memory. That system was also doing per user traffic shaping and NAT. The system felt sluggish at the command line but the users did not complain about performance.
2.1 Requirements
Your system will require two ethernet NIC (Network Interface Card)s. This is because PPPoE must operate on an ethernet interface which does not have an IP address assigned to it. The other NIC will have an IP address assigned to it for access to the network. Multiple PPPoE connections will terminate on the NIC which has no address and they will be forwarded thru the NIC which does have an address assigned. Any Pentium III system with sufficient RAM should easily terminate 100 simultaneous connections. You should use NICs capable of 100 Mbit or Gigabit performance.
2.2 Estimating System Capacity
As described in paragraph 2 above, your system will require about 2 Mbytes of memory and 2 Mhz of CPU per active termination. That is in addition to whatever memory and CPU Linux and the other software running on your server require. It is probably safe to assume that a 500 Mhz Pentium with 512 Mbytes of RAM will terminate about 250 concurrent connections. When in doubt, play it safe. Both CPU and memory are inexpensive. Multi-processor systems would be more desirable in this service because of lower system latencies. Also multiple PPPoE servers are more attractive than a single powerful server because when a server fails the other servers on the network can pick up the load. Redundancy is good.
As it turns out there is a bug either in PPP or in the PPPoE server which will cause problems when the PPP254 virtual device is created. As far as I am aware this is an unavoidable problem unless you wish to modify the actual source code for PPP or PPPoE as appropriate. Therefore you should not expect to terminate more than 255 PPPoE connections on a home brew PPPoE server. Hopefully this problem will be fixed by the developers. I have not mentioned the problem to the developers. Multiple servers running in parallel on a LAN backbone are more reliable anyway. That is how I bypassed this particular limitation.
3. Software
Any recent Linux distribution can be used to build a PPPoE server. Stick with kernel version 2.6 or later. Earlier versions of the software did not ship with versions of PPP which supported RADIUS and therefore support for RADIUS in PAM was required. Later releases of Linux use the more recent versions of the PPP daemon.
Most distributions of Linux will provide the required packages on their installation media. I am using Open SuSE 10.1 in my build and all the packages required are on the installation CDs. I like Open SuSE because kernel mode PPPoE support is built into the distribution. That means that you don’t need to recompile the kernel to use kernel mode. You can also download and build the packages from their respective sites, if you are particularly adept with your flavor of Linux and wish to build using it.
3.1 RP-PPPoE
The Roaring Penguin PPPoE server software is required. Use version 3.5 or later. As mentioned elsewhere this version of the server supports kernel mode PPPoE, when it and the kernel are built for that mode. You can get an RPM for SuSE 10.0 here. The authors of RP-PPPoE claim that the PPPoE server they have released is not really for production use. We have built and operated RP-PPPoE based servers for two years now. They have been robust and have performed quite well in an environment with about 250 active DSL and wireless customers hitting two servers attached directly to our backbone. Note: Since the time of the above writing we have been operating two 750 MHz PIII systems with 512 MBytes RAM to terminate about 400 active broadband customers, still no problems.
3.2 PPPd
The version 2.4 release of the PPP daemon should also be installed on your system. Prior versions may not have support for RADIUS built in to them. This version supports per user configuration via RADIUS attributes. If you understand RADIUS attributes you will have little problem setting up per user configuration. This HOWTO is not going to cover that until it is more mature. We will only set up user login via RADIUS.
3.3 radiusclient
The radiusclient software provides the configuraton files for the PPP daemon’s RADIUS plugin. On SuSE 10.0 it needs to be installed separately. OpenSuSE 10.0 provides it on the installation media. Other versions of Linux may install it as part of the PPP daemon installation. The SuSE 10.0 RPM can be found here. I am sorry, I do not have a link for the tar.gz file.
4.0 Configuraton
Before configuring your PPPoE server you need to make sure you are logged on as root. You should turn off your firewall, if you have one running.
4.1 radiusclient configuration
The PPP daemon plugin’s use the files which are also used by radiusclient. These files are located in the /etc/radiusclient directory. There are two files which must be configured to make your PPPoE server use a RADIUS server to authenticate users. If you do not want your PPPoE server to authenticate users via RADIUS you can skip this section and go to section 4.2.
Open the text file you find at /etc/radiusclient/serves in an editor and insert the IP address of your RADIUS server and the secret this client will use when talking to the RADIUS server. That file will now look something like this:
#Server Name or Client/Server pair Key
#—————- —————
#portmaster.elemental.net hardlyasecret
#portmaster2.elemental.net donttellanyone
#
# uncomment the following line for simple testing of radlogin
# with radiusd 1.16.1
#localhost/localhost testing123
123.45.67.89 MyRadiusSecret
The other file you need to open in your text editor is /etc/radiusclient/radiusclient.conf. In the RADIUS settings section of that file you need to specify your authserver IP address and your acctserver IP addresse. You should also comment out the existing entries which point to localhost.
4.2 PPPd configuration
PPP is very powerful and complex protocol for building point to point links over just about any medium imaginable. PPPoE is a “wrapper protocol” around the PPPd software that establishes and tears down PPP connections over ethernet. PPPoE calls PPPd once it has the necessary information that PPPd requires to maintain a connection.
The PPPd configuration files reside in /etc/ppp. There are two files which are of interest to us. They are /etc/ppp/options and /etc/ppp/pap-secrets. We will do the minimum necessary to configure the PPP daemon to support PPPoE connections. There are many configurable options in the /etc/ppp/options file. A careful reading of the file would be informative to those who have never operated a PPP server.
Changes to /etc/ppp/options:
You will need to open /etc/ppp/options in your text editor. Most configuration files in *nix use a pound (#) sign at the beginning of a line to indicate the line is a comment. Find the line which reads nologin and change it to read #nologin. Find the line which reads #mru 1492 and uncomment it. Find the line which reads #mtu 1492 and uncomment it. Find the line which reads #noreplacedefaultroute and uncomment it.
Find the line which reads #proxyarp and uncomment it. This is only necessary if you are going to assign some clients public IP addresses. If all clients will be NATted, you do not need proxyarp enabled. The safe choice is to enable proxyarp because at worst it generates an error message in the logs.
Find the line(s) which start with #ms-dns 192.168. This is where you configure your DNS server for your clients. Clients will be assigned the DNS servers specified here. You should uncomment one of these lines and edit it to assign a DNS server to your clients.
Lastly there are two lines which you need to add at the bottom of the /etc/ppp/options file. They are:
plugin radius.so
plugin radattr.so
These two lines tell the PPP daemon to use RADIUS for accounting and authentication. If you are not going to use RADIUS, for example, when you intend to authenticate users against the local password file, you would not add those lines to the end of your options file.
Changes to /etc/ppp/pap-secrets:
Open the /etc/ppp/pap-secrets file in your text editor. Look for the section of the file which looks like this:
# INBOUND CONNECTIONS
#client hostname <password> 192.168.1.1
You will place a wild card under client, host name, and the IP address. This states that any client, with any host name, will be assigned any IP address, if that client provides the correct login information. Change that portion of the pap-secrets file to look like this:
# INBOUND CONNECTIONS
#client hostname <password> 192.168.1.1
* * “” *
This completes the basic configuration of the files in /etc/ppp. These are the minimum required settings to build your PPPoE server.
4.3 RP-PPPoE configuration
RP-PPPoE has a configuration file which resides in /etc/ppp also. It is at /etc/ppp/pppoe-server-options. It needs no change. The PPPoE server daemon is a process which must be running to monitor for attempts to connect. It should be launched at system boot time. I use a script to launch the PPPoE server process and to configure NAT (Network Address Translation) for the non-routeable addresses I use behind my PPPoE server. Here is the script:
#!/bin/bash
# —————————————————-
# Starts the PPPoE server and turns on NAT
# —————————————————-
# MAX is the maximum number of addresses your server
# is allowed to hand out.
MAX=250
# BASE is the lowest IP address your server is allowed
# to hand out.
BASE=10.67.7.1
# NAT is the set of addresses which your server will
# NAT behind it. Other addresses behind your server
# WILL NOT be NATed.
NAT=10.67.7.0/24
# MYIP is the public IP address of this server.
MYIP=65.122.24.7
##########################################
# Here is where the script actually starts executing.
##########################################
# Disable IP spoofing on the external interface.
/usr/sbin/iptables -A INPUT -i eth0 -s $NAT -j DROP
#
# Enable NAT for the private addresses we hand out.
/usr/sbin/iptables -t nat -A POSTROUTING -s $NAT -j SNAT –to-source $MYIP
#
# Launch the server.
/usr/sbin/pppoe-server -T 60 -I eth1 -N $MAX -C PPPoE7 -S PPPoE7 -R $BASE
This script is named pppoe-up. It resides in /etc/rc.d/pppoe-up. You must make a copy of this script and then modify it before it will work for your installation. Set the MAX= value to the maximum number of concurrent connections you want your PPPoE server to accept. Set the BASE= value to the base IP address which you want your PPPoE server to use to start handing out addresses. That address could be 192.168.0.1 or any other IP address routable or not. The PPPoE server will start assigning addresses at this point (in my case) 10.67.7.1 and will hand out up to 250 addresses. Set the NAT= value to the network address definition which will cover your address space. In my case I have set it to a Class-C block based at 10.67.7.0. All addresses which are in that block will be NATed behind my PPPoE server. Set the MYIP= value to the IP address you have assigned to your PPPoE server. This is the address your PPPoE server will use to communicate with the world. All users connected to your PPPoE server will use this address, unless you specify otherwise in your RADIUS server or elsewhere.
The script reads these values and actually consists of only the three lines below the MYIP= value. The first line blocks incoming packets which are using addresses the server believes are behind it. The second line enables NAT for all the non-routeable addresses we are going to use on our PPPoE server. The third line actually launches the PPPoE server.
You should edit the line which reads: /usr/sbin/pppoe-server -T 60 -I eth1 -N $MAX -C PPPoE7 -S PPPoE7 -R $BASE and change the references to PPPoE7 in the -C and -S options. The -C option is the hostname of your access concentrator (PPPoE server). The -S option is the TAG this particular server uses to identify itself to the client system. The -S option is used to ensure that a specific client always connects to a specific server, if there are multiple servers on your network. PPPoE client software can be configured to look for ` “service name”. If the service name it looks for matches the service name in the -S option it will connect to that server.
Once you have created the script and placed it in /usr/sbin/pppoe-up execute the following commands at the shell prompt:
# Change to the directory
cd /etc/rc.d
# Make the file executable.
chmod 755 ./pppoe-up
# Set up the script to run in run levels three and five.
chkconfig pppoe-up 35
4.4 Ethernet Interfaces
The PPPoE server needs to bind with an unnumbered ethernet interface running in promiscuous mode. I have never seen a Linux distribution which automates the creation of such an interface. Using whatever tool your Linux distribution provides bring up both of your ethernet interfaces and assign them both addresses. Give the world facing interface (eth0) a real IP address. Give the client facing interface (eth1) 192.168.1.1. Save those settings. Now change to the /etc/sysconfig/network directory and list the files (ls -l). The listing will look something like this:
PPPoE7:/etc/sysconfig/network # ls -l
total 54
drwxr-xr-x 6 root root 440 2006-01-03 12:11 .
drwxr-xr-x 7 root root 1424 2006-01-03 11:58 ..
-rw-r–r– 1 root root 6979 2005-09-09 12:15 config
-rw-r–r– 1 root root 6705 2006-01-02 15:58 dhcp
-rw-r–r– 1 root root 181 2006-01-03 12:10 ifcfg-eth-id-00:50:bf:43:51:47
-rw-r–r– 1 root root 250 2006-01-03 12:11 ifcfg-eth-id-00:50:bf:43:66:05
-rw-r–r– 1 root root 141 2005-09-09 12:15 ifcfg-lo
-rw-r–r– 1 root root 5779 2005-09-09 12:15 ifcfg.template
drwxr-xr-x 2 root root 48 2005-09-09 12:27 if-down.d
-rw-r–r– 1 root root 239 2005-09-09 12:15 ifroute-lo
drwxr-xr-x 2 root root 88 2005-12-30 11:59 if-up.d
drwx—— 2 root root 48 2005-09-09 12:27 providers
-rw-r–r– 1 root root 25 2006-01-02 15:58 routes
drwxr-xr-x 2 root root 1336 2005-12-30 11:59 scripts
-rw-r–r– 1 root root 5469 2005-12-30 11:57 wireless
PPPoE7:/etc/sysconfig/network #
Notice the two files named ifcfg-eth-id-<mac address of interface> these are the files your system uses to configure thos interfaces at boot time. Now execute the command ifconfig eth1 at the command line. The output will look something like this:
PPPoE7:/etc/sysconfig/network # ifconfig eth1
eth0 Link encap:Ethernet HWaddr 00:50:BF:43:66:05
inet addr:192.168.1.1 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::250:bfff:fe43:6605/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1562715 errors:0 dropped:0 overruns:0 frame:0
TX packets:58595 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:127223033 (121.3 Mb) TX bytes:9415968 (8.9 Mb)
Interrupt:10 Base address:0xc000
PPPoE7:/etc/sysconfig/network #
Notice the Hwaddr for the eth1 interface. Now you know which ifcfg-eth-id-xx:xx:xx:xx:xx:xx file is the one for eth1. Open that file in your text editor. It will look something like this:
BOOTPROTO=’static’
BROADCAST=’192.168.1.255′
IPADDR=’192.168.1.1′
MTU=”
NAME=’Realtek RT8139′
NETMASK=’255.255.255.0′
NETWORK=’192.168.1.0′
REMOTE_IPADDR=”
STARTMODE=’auto’
UNIQUE=’rBUF.IQxIdIhhuH7′
USERCONTROL=’no’
_nm_name=’bus-pci-0000:00:09.0′
Change the BROADCAST address to be empty, leave the single quotes, if they are there. Delete IPADDR line entirely. Change the NETWORK to be empty, leave the single quotes if they are there. Delete the NETMASK line entirely.
The modified file will look something like this:
BOOTPROTO=’static’
BROADCAST=”
MTU=”
NAME=’Realtek RT8139′
NETWORK=”
REMOTE_IPADDR=”
STARTMODE=’auto’
UNIQUE=’JNkJ.IQxIdIhhuH7′
USERCONTROL=’no’
_nm_name=’bus-pci-0000:00:0a.0′
We have now configured the system to bring up both interfaces with the world facing interface at eth0 and the client facing interface at eth1. You should execute the command: /etc/rc.d/network restart. That will take down and restart your network. Now you can issue the command: ifconfig to see that you have something like this:
PPPoE7:/etc/sysconfig/network # ifconfig
eth0 Link encap:Ethernet HWaddr 00:50:BF:43:66:05
inet addr:65.122.24.7 Bcast:65.122.24.255 Mask:255.255.255.0
inet6 addr: fe80::250:bfff:fe43:6605/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:1570208 errors:0 dropped:0 overruns:0 frame:0
TX packets:58630 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:127738167 (121.8 Mb) TX bytes:9425522 (8.9 Mb)
Interrupt:10 Base address:0xc000
eth1 Link encap:Ethernet HWaddr 00:50:BF:43:51:47
inet6 addr: fe80::250:bfff:fe43:5147/64 Scope:Link
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:69 errors:0 dropped:0 overruns:0 frame:0
TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6348 (6.1 Kb) TX bytes:258 (258.0 b)
Interrupt:11 Base address:0xc000
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:646 errors:0 dropped:0 overruns:0 frame:0
TX packets:646 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:46470 (45.3 Kb) TX bytes:46470 (45.3 Kb)
PPPoE7:/etc/sysconfig/network #
Notice above that eth0 is up and has an Ipv4 address. Your interfaces may or may not be up in Ipv6 as these are. That does not matter. Notice that eth1 is up and has no Ipv4 address and no Ipv4 netmask.
4.5 IP Forwarding
Now we are going to turn on Ipv4 forwarding. In the /etc/sysconfig directory. There is a file named sysctl. Edit it with a text editor. You will find a reference in it to IP_FORWARD. The default is “no” change it to “yes” and save the file.
Congratulations! Reboot your system and go configure your RADIUS server. It needs to expect service requests from this host. It needs to know the secret you put in /etc/ppp/servers.
5.0 Trouble shooting
After you reboot your system. Try to connect with a PPPoE client. The PPPoE server and the PPPd daemon generate a substantial bit of logging information, even with their default settings. You will find this information in /var/log/messages. There are options to increase the logging level in the /etc/ppp/options file. If you wish to increase that look for “debug” in that file.
If you find that you can get a PPPoE connection but you can not go anywhere, have a look at the value stored in /proc/sys/net/ipv4/ip_forward. If it is 1 you are configured for forwarding Ipv4 traffic. If it is 0 you are not configured for forwarging Ipv4 traffic. Check that your firewall is not modifying this value. I recommend you ensure that your firewall is turned off on the unit until you are certain you have everything working correctly. The value of IP_FORWARD in the SuSE firewall overrides the value in /etc/sysconfic/sysctl.