{"id":141,"date":"2013-03-14T17:18:50","date_gmt":"2013-03-14T17:18:50","guid":{"rendered":"http:\/\/asdfghjkl.me.uk\/blog\/?p=141"},"modified":"2013-12-01T21:33:46","modified_gmt":"2013-12-01T21:33:46","slug":"xenserveriscool","status":"publish","type":"post","link":"https:\/\/asdfghjkl.me.uk\/blog\/xenserveriscool\/","title":{"rendered":"So many VMs, so few IPs"},"content":{"rendered":"<p>I share a dedicated server with a few friends. Aside from being a fun learning experience (!), this saves a significant amount of money - for about \u00a310\/month I get 500GB disk space, 2GB RAM, 2 CPU cores and (essentially) as much bandwidth as I can eat.<\/p>\n<p><!--more--><\/p>\n<p><a href=\"https:\/\/asdfghjkl.me.uk\/blog\/wp-content\/uploads\/xencenter.png\"><img decoding=\"async\" loading=\"lazy\" class=\"aligncenter size-medium wp-image-142\" alt=\"XenCenter\" src=\"https:\/\/asdfghjkl.me.uk\/blog\/wp-content\/uploads\/xencenter-300x209.png\" width=\"300\" height=\"209\" srcset=\"https:\/\/asdfghjkl.me.uk\/blog\/wp-content\/uploads\/xencenter-300x209.png 300w, https:\/\/asdfghjkl.me.uk\/blog\/wp-content\/uploads\/xencenter-624x435.png 624w, https:\/\/asdfghjkl.me.uk\/blog\/wp-content\/uploads\/xencenter.png 994w\" sizes=\"(max-width: 300px) 100vw, 300px\" \/><\/a><\/p>\n<p>Problem is, the provider only gives us a limited number of IPv4 addresses, so we have to share. Since we're using XenServer, I've set up a tiny router VM for the other VMs to connect to. This bit was simple - I set up an external network bound to the external IP address, and an internal network for the VMs. Here's what my <em>\/etc\/network\/interfaces<\/em>\u00a0looks like:<\/p>\n<pre># loopback\r\nauto lo\r\niface lo inet loopback\r\n\r\n# external network\r\nallow-hotplug eth0\r\niface eth0 inet static\r\n    address EXTERNAL_IP\r\n    netmask 255.255.255.255\r\n    broadcast EXTERNAL_IP\r\n    dns-nameservers 8.8.8.8\r\n    dns-search asdfghjkl.me.uk\r\n\r\n# internal network\r\nauto eth1\r\niface eth1 inet static\r\n    address 192.168.1.254\r\n    netmask 255.255.255.0<\/pre>\n<p>I then created a set of\u00a0<a href=\"https:\/\/en.wikipedia.org\/wiki\/Iptables\">iptables<\/a>\u00a0rules to forward all traffic from the VMs to the outside world:<\/p>\n<pre># clear existing rules:\r\niptables --flush\r\niptables -t nat --flush\r\niptables --delete-chain\r\n# forward all internal traffic (eth1) to the external network (eth0):\r\niptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE\r\niptables -A FORWARD -i eth1 -o eth0 -j ACCEPT<\/pre>\n<p>Like on a home router, port forwarding is used to direct traffic to the relevant VM.<\/p>\n<pre># tcp traffic:\r\niptables -t nat -A PREROUTING -p tcp -i eth0 --dport 1234 -j DNAT --to-destination 192.168.1.1:1234\r\niptables -A FORWARD -p tcp -d 192.168.1.4 --dport 1234 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT\r\n\r\n# udp traffic:\r\niptables -t nat -A PREROUTING -p udp -i eth0 --dport 1235 -j DNAT --to-destination 192.168.1.2:1235\r\niptables -A FORWARD -p tcp -d 192.168.1.2 --dport 1235 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT\r\n\r\n# multiple ports:\r\niptables -t nat -A PREROUTING -p tcp -i eth0 --dport 1236:1240 -j DNAT --to-destination 192.168.1.3\r\niptables -A FORWARD -p tcp -d 192.168.1.3 --match multiport --dports 1236:1240 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT<\/pre>\n<p>To apply the firewall rules on startup, I added the script to <em>\/etc\/rc.local<\/em>.<\/p>\n<p>For hosting game servers, this is all well and good, as different games tend to use different ports. But for webhosting, everyone wants port 80 - addresses like http:\/\/asdfghjkl.me.uk:1234 look unprofessional! To achieve this, I set up apache on the router to proxy requests for specific domains to their relevant VM. Here's an example VirtualHost directive:<\/p>\n<pre>&lt;VirtualHost EXTERNAL_IP:80&gt;\r\nServerName asdfghjkl.me.uk\r\nServerAlias asdfghjkl.me.uk *.asdfghjkl.me.uk\r\nRewriteEngine on\r\nProxyRequests off\r\nProxyPreserveHost on\r\nProxyPass \/ http:\/\/192.168.1.1:80\/\r\nProxyPassReverse \/ http:\/\/192.168.1.1:80\/\r\n&lt;\/VirtualHost&gt;<\/pre>\n<p>This will redirect traffic for asdfghjkl.me.uk and all subdomains to my VM. Without ProxyPreserveHost, the Host: line passed to the VM would always be set to 192.168.1.1, which would make name-based virtual hosting (e.g. for subdomains) impossible. The other issue with this setup is that the VM's access log will only contain the router's address for every request. Luckily the originating IP is sent in the X-Forwarded-For header, and an apache module is available to log this instead. On the VM:<\/p>\n<pre>sudo apt-get install libapache2-mod-rpaf<br id=\".reactRoot[250].[1][2][1]{comment171939846284026_172004876277523}.0.[1].0.[1].0.[0].[0][2].0.[7]\" \/>sudo a2enmod rpaf<br id=\".reactRoot[250].[1][2][1]{comment171939846284026_172004876277523}.0.[1].0.[1].0.[0].[0][2].0.[9]\" \/>sudo nano \/etc\/apache2\/mods-available\/rpaf.conf<\/pre>\n<p>Add the following lines (where\u00a0192.168.1.254 is the router's address):<\/p>\n<pre>RPAFenable On<br id=\".reactRoot[250].[1][2][1]{comment171939846284026_172004876277523}.0.[1].0.[1].0.[0].[0][2].0.[16]\" \/>RPAFsethostname On<br id=\".reactRoot[250].[1][2][1]{comment171939846284026_172004876277523}.0.[1].0.[1].0.[0].[0][2].0.[18]\" \/>RPAFproxy_ips 192.168.1.254<\/pre>\n<p>...and run <em>sudo invoke-rc.d apache2 restart\u00a0<\/em>to apply the changes.\u00a0It's worth noting that requests without a Host: line will be sent to the first site available (alphabetical if you have each VirtualHost directive in a separate file, or first in the list if you don't). I set up a dummy site with a bit of HTML to avoid this.<\/p>\n<p>This all works fine for HTTP traffic, but HTTPS is slightly more involved. The name-based virtual hosts which we're using here will give certificate errors, as the SSL handshake takes place before the Host: line is sent. You either need a unique IP or a unique port for each site. Luckily, most browsers (a notable exception being IE8 or below on Windows XP) support <a href=\"https:\/\/en.wikipedia.org\/wiki\/Server_Name_Indication\">Server Name Indication<\/a>, which sends the hostname at the start of the handshaking process. We enable mod_ssl on the server with <em>sudo a2enmod ssl<\/em>, then configure each site with a VirtualHost directive as before:<\/p>\n<pre>&lt;IfModule mod_ssl.c&gt;\r\nSSLStrictSNIVHostCheck off\r\n\r\n&lt;VirtualHost EXTERNAL_IP:443&gt;\r\nDocumentRoot \/var\/www\/nohostname-ssl\r\nServerName subdomain.asdfghjkl.me.uk\r\n\r\nSSLEngine on\r\nSSLCertificateFile \/etc\/ssl\/certs\/certificate1.crt\r\nSSLCertificateKeyFile \/etc\/ssl\/private\/certificate1.key\r\nSSLCertificateChainFile \/etc\/ssl\/certs\/chain.pem\r\nSSLCACertificateFile \/etc\/ssl\/certs\/ca.pem\r\n&lt;\/VirtualHost&gt;\r\n\r\n&lt;VirtualHost EXTERNAL_IP:443&gt;\r\nServerName asdfghjkl.me.uk\r\nServerAlias asdfghjkl.me.uk *.asdfghjkl.me.uk\r\nRewriteEngine on\r\nProxyRequests off\r\nProxyPreserveHost on\r\nProxyPass \/ http:\/\/192.168.1.1:80\/\r\nProxyPassReverse \/ http:\/\/192.168.1.1:80\/\r\n\r\nSSLEngine on\r\nSSLCertificateFile \/etc\/ssl\/certs\/certificate2.crt\r\nSSLCertificateKeyFile \/etc\/ssl\/private\/certificate2.key\r\nSSLCertificateChainFile \/etc\/ssl\/certs\/chain.pem\r\nSSLCACertificateFile \/etc\/ssl\/certs\/ca.pem\r\n&lt;\/VirtualHost&gt;\r\n&lt;\/IfModule&gt;<\/pre>\n<p>Disabling SSLStrictSNIVHostCheck means the first site in the list will be used if no hostname is specified, or if the browser doesn't support SNI. There'll still be a certificate error in the latter case, but it's better than nothing! As before, I've set up a small dummy site explaining that a browser upgrade is necessary.<\/p>\n<p>I noticed soon after setting this up that websites behind the proxy were occasionally giving the error \u00a0\"The proxy server received an invalid response from an upstream server. The proxy server could not handle the request GET \/\". The solution, as found <a href=\"http:\/\/serverfault.com\/questions\/206738\/intermittent-error-when-using-mod-proxy-to-do-reverse-proxy-to-soap-service\">here<\/a>, was to add the following lines to\u00a0<em>\/etc\/apache2\/apache2.conf<\/em>:<\/p>\n<pre><code>SetEnv force-proxy-request-1.0 1\r\nSetEnv proxy-nokeepalive 1<\/code><\/pre>\n<p>If you want to force SSL for a given site, you'd usually replace the proxy lines for the non-SSL vhost with something like \"Redirect permanent \/ https:\/\/asdfghjkl.me.uk\". This works for a single site, but means all subdomains would also be redirected there - not ideal! Instead, you can use mod_rewrite to redirect based on the hostname:<\/p>\n<pre>RewriteEngine On\r\nRewriteCond %{HTTPS} !=on\r\nRewriteRule ^\/?(.*) https:\/\/%{HTTP_HOST}\/$1 [R,L]<\/pre>\n<p>You'll need a wildcard certificate (i.e. *.domain) for this to work properly, or a separate VirtualHost directive for each subdomain. Up to you really. As always, leave a comment or email me at hello at this domain if you're having any problems!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I share a dedicated server with a few friends. Aside from being a fun learning experience (!), this saves a significant amount of money - for about \u00a310\/month I get 500GB disk space, 2GB RAM, 2 CPU cores and (essentially) as much bandwidth as I can eat.<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[6,7],"tags":[],"_links":{"self":[{"href":"https:\/\/asdfghjkl.me.uk\/blog\/wp-json\/wp\/v2\/posts\/141"}],"collection":[{"href":"https:\/\/asdfghjkl.me.uk\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/asdfghjkl.me.uk\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/asdfghjkl.me.uk\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/asdfghjkl.me.uk\/blog\/wp-json\/wp\/v2\/comments?post=141"}],"version-history":[{"count":11,"href":"https:\/\/asdfghjkl.me.uk\/blog\/wp-json\/wp\/v2\/posts\/141\/revisions"}],"predecessor-version":[{"id":185,"href":"https:\/\/asdfghjkl.me.uk\/blog\/wp-json\/wp\/v2\/posts\/141\/revisions\/185"}],"wp:attachment":[{"href":"https:\/\/asdfghjkl.me.uk\/blog\/wp-json\/wp\/v2\/media?parent=141"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/asdfghjkl.me.uk\/blog\/wp-json\/wp\/v2\/categories?post=141"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/asdfghjkl.me.uk\/blog\/wp-json\/wp\/v2\/tags?post=141"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}