Internet connection sharing through a computer with two NIC's

Recently a new server room was built at the office and now all the network and server gear has been moved there. I used to use wireless connections on my notebooks out of convenience, but now that the wireless router is inside the server room instead of in the middle of an open floor plan office, the wireless signal is too weak.

My notebooks are connected to the internal network, however. Most of the test servers have dual NIC's so on a server with Fedora 23 installed, I first enabled port forwarding manually:

# check if port forwarding is enabled (it isn't)

[fedgro@fx8350no2 sysctl.d]$ sudo sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 0

# enable it

[fedgro@fx8350no2 sysctl.d]$ sudo sysctl net.ipv4.ip_forward=1
net.ipv4.ip_forward = 1

# verify that port forwarding has been enabled:

[fedgro@fx8350no2 sysctl.d]$ sudo sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

The change above will only apply until the next reboot, so to make the changes permanent, I want to edit sysctl.conf

In recent versions of Fedora using systemd, this conf file is no longer monolithic and is generated at boot by systemd from files in /usr/lib/sysctl.d/ (distribution default kernel settings) and in /etc/sysctl.d/ (custom kernel settings)

Here is the content of each directory:

[fedgro@fx8350no2 sysctl.d]$ cd /usr/lib/sysctl.d/
[fedgro@fx8350no2 sysctl.d]$ ls
00-system.conf              50-coredump.conf  60-libvirtd.conf
10-default-yama-scope.conf  50-default.conf
[fedgro@fx8350no2 sysctl.d]$ cd /etc/sysctl.d/
[fedgro@fx8350no2 sysctl.d]$ ls
50-libreswan.conf  99-sysctl.conf

Under /etc/sysctl.d I created a custom file 98-ipv4-forward.conf containing the single line

net.ipv4.ip_forward=1

Now that ipv4 port forwarding has been enabled, it is time to set up IP masquerading in the firewall. Fedora 23 uses firewalld instead of iptables, so to make the appropriate settings you can use the GUI firewall-config or the cli tool firewall-cmd.

You can see that firewalld is running:

[fedgro@fx8350no2 sysctl.d]$ systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
   Active: active (running) since Mon 2016-02-22 10:13:28 KST; 23h ago
 Main PID: 2132 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─2132 /usr/bin/python3 -Es /usr/sbin/firewalld --nofork --nopid

Feb 22 10:13:28 fx8350no2 systemd[1]: Starting firewalld - dynamic firewal.....
Feb 22 10:13:28 fx8350no2 systemd[1]: Started firewalld - dynamic firewall...n.
Hint: Some lines were ellipsized, use -l to show in full.

Now I use firewall-cmd below to set up masquerading:

# Check the default firewall zone
# (FedoraServer is the default for F23)
[fedgro@fx8350no2 sysctl.d]$ sudo firewall-cmd --get-default
FedoraServer

Check the firewall zone for the NIC connected to a router providing access to the Internet:

[fedgro@fx8350no2 sysctl.d]$ sudo firewall-cmd --get-zone-of-interface=enp5s0
FedoraServer
 
Add IP masquerading to the default zone:

[fedgro@fx8350no2 sysctl.d]$ sudo firewall-cmd --zone=FedoraServer --add-masquerade
success

[fedgro@fx8350no2 sysctl.d]$ sudo firewall-cmd --permanent --zone=FedoraServer --add-masquerade

success

Verify changes have been made to FedoraServer zone:
[fedgro@fx8350no2 sysctl.d]$ firewall-cmd --zone=FedoraServer --list-all
FedoraServer (default, active)
  interfaces: enp5s0
  sources:
  services: cockpit dhcpv6-client http https ipp ipp-client samba-client sane ssh
  ports: 80/tcp 631/tcp
  protocols:
  masquerade: yes
  forward-ports:
  icmp-blocks:
  rich rules:

Now from my notebook connected to the internal network, I simply add the following route (to the server with dual NIC's):

sudo ip r add default via 192.168.95.145 dev enp1s0

where 192.168.95.145 is the IP of the NIC connected to the internal network on the dual-NIC server, and where enp1s0 is the ethernet port on my notebook.

And now the notebook connected to the internal network can talk to the outside world. Let's test on Google's DNS server 8.8.8.8:

[archjun@pinkS310 ~]$ ping 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=51 time=36.7 ms
64 bytes from 8.8.8.8: icmp_seq=2 ttl=51 time=35.3 ms
64 bytes from 8.8.8.8: icmp_seq=3 ttl=51 time=36.1 ms
64 bytes from 8.8.8.8: icmp_seq=4 ttl=51 time=36.6 ms
64 bytes from 8.8.8.8: icmp_seq=5 ttl=51 time=35.9 ms
^C
--- 8.8.8.8 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4006ms
rtt min/avg/max/mdev = 35.342/36.157/36.718/0.496 ms
[archjun@pinkS310 ~]$ traceroute 8.8.8.8
traceroute to 8.8.8.8 (8.8.8.8), 30 hops max, 60 byte packets
 1  gateway (192.168.95.145)  0.299 ms  0.278 ms  0.279 ms
 2  192.168.30.1 (192.168.30.1)  0.657 ms  0.695 ms  0.743 ms
 3  REDACTED (REDACTED)  11.346 ms  11.357 ms  11.349 ms
 4  REDACTED (REDACTED)  9.661 ms  13.730 ms  13.750 ms
 5  REDACTED (REDACTED)  9.541 ms  9.571 ms  9.566 ms
 6  112.189.28.97 (112.189.28.97)  8.167 ms  6.349 ms 112.189.29.97 (112.189.29.97)  15.201 ms
 7  112.174.103.225 (112.174.103.225)  10.270 ms 112.174.103.213 (112.174.103.213)  8.211 ms 112.174.119.205 (112.174.119.205)  6.721 ms
 8  112.174.48.162 (112.174.48.162)  6.676 ms  6.688 ms 112.174.8.42 (112.174.8.42)  6.898 ms
 9  112.174.83.218 (112.174.83.218)  6.854 ms 112.174.83.34 (112.174.83.34)  6.857 ms 112.174.84.122 (112.174.84.122)  6.957 ms
10  72.14.194.106 (72.14.194.106)  36.267 ms  36.246 ms  36.211 ms
11  216.239.54.13 (216.239.54.13)  36.528 ms 216.239.50.241 (216.239.50.241)  36.149 ms 216.239.54.13 (216.239.54.13)  41.606 ms
12  209.85.254.17 (209.85.254.17)  34.989 ms 209.85.246.89 (209.85.246.89)  37.053 ms 209.85.255.241 (209.85.255.241)  36.305 ms
13  google-public-dns-a.google.com (8.8.8.8)  36.593 ms  36.592 ms  42.129 ms

You can see that the first hop in the traceroute is to 192.168.95.145, the server on the internal network which has 2 NIC's. 1 NIC is connected to the internal network while one NIC is connected to the external network. It forwards the packets from the internal 192.168.95.0/24 network to the external network on subnet 192.168.30.0/24 which has access to the Internet through a router.

References:
https://wiki.archlinux.org/index.php/Internet_sharing
http://www.rudraraj.net/2014/12/21/internet-connection-sharing-ics-on-centos-7/

(the second link provides some useful commands for firewall-cmd)

Setup ibus-hangul in Fedora 23

Fedora 23 XFCE 데스크탑 판을 최근에 설치 했는데 한글 입력 방식 설치하기 위해

sudo dnf install ibus-hangul

명령어를 실행했습니다. 의존 패키지 ibus까지 다 설치 되고

ibus-daemon -rdx &

명령을 터미널에 실행하여 ibus 데몬은 배경에서 돌아가게 했습니다. ibus 설정에서 추가 언어 한글을 세팅 했는데도 한글 모드로 전환할 때 한글 입력이 안 됐습니다.

pidof ibus-daemon 하여 pid 알아낸 다음 sudo kill -15 xxx 해서 ibus-daemon을 멈추게 하고 문제를 추려내기 위해 ibus 데몬을 -v (또는 --verbose) 옵션으로 다시 실행했습니다. 구지 pid 킬 안 해도 -r (또는 --replace) 옵션으로 ibus-daemon을 실행시키면 기존에 돌고 있는 ibus 데몬 프로세스를 죽이고 새로운 인스턴스를 실행한답니다.

ibus-daemon -rvx

명령 실행하니까 vala와 gtk 경고나 에러가 나타났습니다. 혹시나 해서 xfce4-vala (제가 쓰는 DE는 xfce4이기 때문), ibus-pygtk2와 ibus-qt를 설치했습니다. X 윈도우 세션에서 로그아웃 하기 전에 달라진 게 없었지만 GUI환경에서 로그아웃 로긴 한 다음에 한글 입력이 잘 됐습니다!

다양한 앱 그리고 Java 프로그램에서까지 ibus-hangul 입력을 원하시면 제가 옛날 올린 글을 참조 하시길 바랍니다 (.bashrc 설정과 Java 앱 실행 스크립트 예제):

http://eatpeppershothot.blogspot.kr/2014/03/enabling-ibus-support-in-omegat.html

-------------------------------------------------------------

To get ibus-hangul working in Fedora 23, keep in mind that it is not sufficient to just dnf install ibus-hangul . There are a couple of additional packages you need to install:

ibus-pygtk2
ibus-qt

In my recollection, ibus-gtk2 and ibus-gtk3 are installed by default together with ibus-hangul, but if they aren't, make sure to install them as well.

Once this is done, log out of your X windows session and log back in (or just reboot) and you should find that Korean and Chinese (hanja) input works fine. Keep in mind that ibus-daemon needs to be running and that you have added Korean as an additional language in the ibus preferences.

Manually Installing xfce4 on Fedora 23 Server

I recently found myself needing to launch a VM image in qcow2 format using KVM. The VM required various additional settings such as bridged networking (using one of the host's ethernet ports) and attachment of virtual storage (/dev/vdXY). Unfortunately, since I am not familiar with the libvirt cli tool virsh, I had to rely on the nice GUI interface of virt-manager. The server machine I was working on did not have X windows installed, so I assumed that simply invoking

dnf install @xfce-desktop

would be sufficient. After installing all the packages and dependencies for xfce4, when I tried to launch X windows with startxfce4, I got the following error:

xinit: unable to run server "X": No such file or directory

The problem is caused by xfce4 packages not having X windows as a dependency!

To manually install all the X windows packages in Fedora 23, invoke the following:

dnf group install base-x

This will install all the all the xorg-x11-[blah] packages necessary for X windows to start.

UEFI and Legacy BIOS PXE netboot through dnsmasq

Nowadays new computers for both consumers and the server room support UEFI (Unified Extensible Firmware Interface) and I have even seen some recent machines that don't support Legacy BIOS at all.

Legacy BIOS PXE netboot uses a UNDI (Unified Network Device Interface) driver but UEFI PXE does not. I therefore had to make some changes in my /etc/dnsmasq.conf file to enable UEFI PXE. Luckily, dnsmasq can autodetect the type of pxe client it is dealing with if you set a few options in the .conf file:

# Test for the architecture of a netboot client. PXE clients are
# supposed to send their architecture as option 93. (See RFC 4578)
dhcp-match=x86PC, option:client-arch, 0 #BIOS x86
dhcp-match=BC_EFI, option:client-arch, 7 #EFI x86-64

# Load different PXE boot image depending on client architecture
pxe-service=tag:x86PC,X86PC, "Install Linux on x86 legacy BIOS", pxelinux
pxe-service=tag:BC_EFI,BC_EFI, "Install Linux on x86-64 UEFI", uefi/bootx64.efi


# Set boot file name only when tag is "bios" or "uefi"
dhcp-boot=tag:x86PC,pxelinux.0  # for Legacy BIOS detected by dhcp-match above
dhcp-boot=tag:BC_EFI,uefi/bootx64.efi # for UEFI arch detected by dhcp-match above

# Enable dnsmasq's built-in TFTP server
enable-tftp

# Set the root directory for files available via FTP.
tftp-root=/usr/local/tftpboot/pxelinux

My entire dnsmasq.conf file can be found here:
https://github.com/gojun077/jun-dotfiles/blob/master/dnsmasq

The directory hierarchy of my tftp-root directory is as follows:

[archjun@pinkS310 pxelinux]$ tree -d /usr/local/tftpboot/pxelinux/
/usr/local/tftpboot/pxelinux/
├── EFI
│   └── BOOT
├── images
│   ├── arch_x64
│   ├── fedora23_x64
│   └── xubuntu15-10_x64
├── kickstart-pxe
├── pxelinux.cfg
└── uefi

9 directories

Note that this hierarchy will be different depending on which Linux distribution's bootx64.efi file you decide to use. Although the EFI boot files are named almost identically across various linux distros, bootx64.efi looks for boot-related files in slightly different paths. You can find this file (and other EFI boot-related files) from the installation .iso file for your distribution. Normally the files will be located in the path ../EFI/BOOT/ on the installation DVD/iso image and will contain the files bootx64.efi (BOOTX64.EFI), grubx64.efi, a sample grub.cfg and possibly other files (like MokManager.efi and shim.efi for UEFI secure boot).

Since I am using the bootx64.efi file from CentOS 7,  I have to place files in the directories that CentOS 7 expects. Recall that my tftp-root= path defined in dnsmasq.conf is /usr/local/tftpboot/pxelinux/, hereafter simply referred to as tftp-root. The required paths are as follows.

EFI network boot program (NBP): tftp-root/uefi/bootx64.efi
EFI grub loader: tftp-root/grubx64.efi,
EFI grub menu: tftp-root/grub.cfg or tftp-root/EFI/BOOT/grub.cfg

The paths will be different if you use a bootx64.efi from another distro, however. For example, Fedora 23 looks for grubx64.efi within tftp-root/uefi/ while Ubuntu looks for grubx64.efi under tftp-root/.

The nice thing about the bootx64.efi from CentOS 7 is that it emits verbose debugging info in the systemd journal (which can be viewed in real-time with journalctl -f) if it cannot find boot files in the expected paths. Here is an example from journalctl:

...
Feb 01 17:38:06 pinkS310 dnsmasq-tftp[31773]: sent /usr/local/tftpboot/pxelinux/uefi/bootx64.efi to 192.168.95.150
Feb 01 17:38:06 pinkS310 dnsmasq-tftp[31773]: file /usr/local/tftpboot/pxelinux/grubx64.efi not found
...

By looking carefully at the system logs in real-time, you can find out where you need to place files in your tftp-root directory hierarchy, but I found that verbose debugging is not emitted when using the bootx64.efi files from Fedora 23 and Ubuntu.

Although I am using the CentOS 7 EFI boot files, I can install non-CentOS distros over PXE through grub menu once grub.cfg is loaded.

Here is my grub.cfg which installs Fedora 23 Server over http and allows me to control the install from a remote machine using vncviewer:



You will notice that the kernel vmlinuz and the initial ramdisk initrd.img  are in the images directory which is a subdirectory of tftp-root (refer to the directory tree above). This subdirectory contains more directories storing the kernels and ramdisk images for multiple distros (in my case, fedora, archlinux, and ubuntu). These images can be used over both UEFI and legacy BIOS PXE.

If dnsmasq detects that a client only supports legacy BIOS PXE when it receives a DHCP request, it will send pxelinux.0 instead of bootx64.efi as the NBP. Then instead of grub.cfg, the default legacy BIOS PXE config file will be loaded from tftp-root/pxelinux.cfg/default

Here is a sample legacy BIOS pxe boot menu for various Linux distros:

https://github.com/gojun077/pxe-config/blob/master/pxelinux.cfg/default

Note that the config syntax differs by linux distribution. In the case of Archlinux, before loading the initial ramdisk, you must first load Intel microcode (if you are using an Intel CPU). Also the syntax for sending installation files over http differs from that of, say, Fedora or RHEL/CentOS.