Residential Mail Server Hosting for Fun & Profit

Long, Long ago, I ran a mail server for myself and others. I even earned a small profit. 

 This server ran on a Sun SparcStation using standard unix tools, which sat in my house plugged into a UPS just in case the power went out.  

I had a commercial DSL connection which was slow by today's standards - but the router never needed to be rebooted, and I had several static IP addresses. 

 The setup was simple: 

1. Purchase a comain 

2. Set up DNS with an MX record which pointed to your mail server 

 At this point, I could already send and receive emails from the server.   

To allow people who didn't love using PINE over SSH to use it, you just had to run IMAPd and such to allow them to connect via Eudora, Outlook, Thunderbird, or whatever client they wanted to use. 

 When web mail became popular, I even set up Roundcube Mail.  

Despite the average penetration and speed of the Internet increasing, this setup became less and less tenable as the years went by. 

Now? I have a 2Gbps connection. I could easily upgrade to 10Gbps but there is little point when that would also mean upgrading my switches, etc. The problem is multifauceted. 

There are actually three major problems: 

1. Spammers 

2. Dynamic IPs 

3. NAT 

 Human nature being what it is, spam quickly became a problem on the internet, and people have become less and less tolerant of it over time.  

Spam

It seems people are tired of hearing from Nigerian princes. 

In order to prevent people from sending out spam from residential connections, many (if not most) residential internet providers have blocked outbound connections from port 25 (which is where SMTP, the language of email sending, lives).  You can't simply use a different port when sending, because the remote end (which you don't control) is listening on port 25.  

 Even if your provider hasn't blocked port 25, there is a strong chance that your address is on one of the "Residential Subscriber Lists", which lists all of IP address ranges used by internet service providers for residential connections. Mail servers operated by large companies typically use these lists to determine if your IP address is on a residential connection, and if it is then they simply block any attempt from your server to deliver mail to them.  In fact, I also subscribed to these lists.  They didn't block me because I was on a commercial connection.  

Some ISPs even block Inbound connections to port 25, such that you can't even receive email directly on a residential connection.  Obviously, you can control the port that your mail server listens on - but everyone will still try to send mail to your server on the standard port 25, so if you don't use that you won't receive anything.  

You can use tools like Apple Mail, Thunderbird, etc. to receive email because they use protocols POP or IMAP to pull email from the mail provider you subscribe to.  Likewise, sending email these days is tightly controlled.  Your ISP might allow ports 587 or 465, which always require authentication and thus make it difficult to send spam - but that basically means you need to send email to an intermediate server operated by an external mail provider, cutting off self-hosting.  

Dynamic IP Addresses

Originally the internet was designed with static IP addresses in mind.  An IP address was to be like a phone number.  You would have one assigned, and then people would know they could reach you at that number.  

When consumer internet started to gain popularity, it was first over dial-up connections.  These connections aren't constant, but only last a few minutes to hours.  You needed to disconnect when you wanted to make a phone call.  The internet service providers figured out that they didn't need a different IP address per customer, just one per modem.  

If you dialed in today you might get one address, but when you dial in tomorrow you would probably get a different IP address.  

If your mail use case was web browsing, then this didn't matter so much.  Basically speaking, connections to web servers always originate from the client, and so only the Ip address of the web server needs to remain constant.  Web servers aren't picky about client addresses, so even if your IP address changes once in a while, that's okay.  

But if you want to run a mail server (or even web server) then when your IP address suddenly changes, suddenly people can't reach you.  To further the analogy, they're trying to call your old phone number even though your number has changed.  

There are work-arounds for this, the main one being so-called "Dynamic DNS".  This basically works by having your computer update the DNS server.  The DNS server checks your new IP address and updates the records to reflect this.  Then when people look up your IP address, they will get the correct (new) address.  

This isn't a perfect solution, as it will take some time for your computer or router to notice that your IP address has changed and update the DNS server.  Once the DNS server has been updated, some clients and caching DNS servers inbetween will also take time to take notice of the change.  

This means that every time your IP address changes, there could be people who can't connect to you for several minutes.  This makes operating servers on dynamic IP addresses less than optimal for mission critical high volume applications.  

Mail is actually slightly forgiving here because if delivery fails, the sending mail server will typically try again later.  

Network Address Translation (NAT)

This is, to me, the most evil of them all.  

Because there aren't enough IP addresses to go around, ISPs tpically only allocate one per household.  So even if you have a laptop, a smart phone, a server, a tablet, and a bunch of IOT gadgets, your house just gets 1 single IP address.  

The idea is this, your gadgets will all get a "local" address that they can use to communicate with each other, but when they need to talk to the outside world, they need to communicate through your router (which holds the real IP address).  

This is a bit complicated, because the router had to figure out the destination for data coming in.  As an example, let's say you are on your laptop and you request a photo of Tokyo Tower from google.  Your PC asks the router to send this request out to google, which it duitifully does.  This is easy enough because Google has a real (global, public) IP address.  

Later, Google sends back the image file, but as far as Google knows, your router is the one who asked for it.  The router receives this data from Google, but has to decide the final destination.  It actually has to maintain a list of who asked for what, when, so that it can direct the replies appropriately.  This is called the NAT table, but it allows the router to know that the reply from Google is destined for my laptop instead of my sister's iPhone or the WiFi enabled microwave oven.  

If you think about it, this means that the router builts the NAT table up from outgoing requests from devices on the network to the public internet.  If a request suddenly came in from the outside that was not a response to a packet sent seconds ago, then the router had no idea who it is for.  

This is the case for server applications like a Web Server or Mail Server.  A request suddenly comes into the router asking for a web page or trying to connect to port 25 - but the router looks in the NAT table and doesn't find any matching entries, so it doesn't know where to send the incoming data and simply discards it.  

There are a few ways around this:

1. Port Forwarding - This is when you set up a special rule that says "Any traffic destined for port 25 on this network always goes to this machine".  You can set up a rule like this for a web server, a mail server, etc.  You can only have one machine per port - but it's better than nothing.

2. DMZ - This allows you to specify a single machine inside the network to receive all incoming traffic that the router doesn't know where else to route.  This could be your "server" if you are running multiple different services and don't want to create lots of separate port forwarding rules.  

3. Various Dynamic Port Forwarding Schemes - These were developed to allow things like Skype and some video game chat protocols, etc. to work properly.  Basically, they allow a PC to talk to your router and request port forwarding rules to be added on the fly.  They are usually temporary rules.

With IP v6, there are more Ip addresses than we could ever think of using, but deployment and support is still spotty.  Also, using NAT is often seen as a security benefit these days.  After all, even if you haven't properly set up firewalls on your machines external requests will be dropped.  

Problem Summary

Basically, this all adds up to a very hostile environment to try to run any server in - at least if the goal is to have the server open to the internet at large.  For private access, you can use VPN tools like TailScale, but something like a Blog, Mail Server, or Photo Sharing site isn't much good if only you can access it.  

For the average home user, then, even if they set up a mail server, it will be behind NAT.  Even if they set up port forwarding, it will be on a dynamic IP.  Even if they set up DDNS, it will be actively blocked by most major mail servers, and possibly by your ISP.  Combine all of this together, and it seems hopeless.

Recent Developments

I've read about stuff like MailCow and Stalwart, but when I really got interested was when I started playing with MailPlus (which is part of DSM from Synology).  The reason I likes MailPlus was because it had not only a server, but also web mail and mobile components.  If you use MailPlus, you can use the client of your choice (like Thunderbird or Apple Mail), the Synology MailPlus app, and/or the MailPlus web app.  The look and feel of the web app is similar to Gmail and other webmail apps.  The system also has calendar and address book functionality, and is simple and lightweight.  

I started using MailPlus by running the server entirely locally, and using the import function to import emails from Gmail.  This works not only with Gmail, but also over IMAP, etc., so you can migrate a large volume of your mail to free up space and just use MailPlus as an archive to keep your old mails.  

Likewise, you can migrate from ProtonMail ising IMAP if you use their local bridge software.  

Recently, I learned about Cloudflare's mail routing feature.  Combined with their mail sending functionality (or that of ReMail or other vendors), you can actually send and receive email indirectly using servers that do have trusted public commercial IP addresses.  

Specifically: 

1. You can set up ReSend to allow you to send outgoing email from your domain, and then set up MailPlus to use ReSend for sending.  Postmark is another reputable option.  (You should also be able to use CloudFlare email sending for this, however I have not tested this option yet).  

2. You can set up CloudFlare email routing to receive email on Domain A and forward all such email to an email on Domain B.  The publicly visible domain A will receive emails from any normal server and should never be down - so everyone will think they have delivered emails to your successfully.  Cloudflare will attempt to deliver your emails on domain B by contacting your email server directly if it is set up in DNS correctly.  If it's down or unreachable, CloudFlare will retry the delivery.  

This means that short downtimes and minor blips in your server's availability are invisible to the internet, and you can receive emails reliably.  You can send emails reliably as well, since they aren't seen as coming directly from your server on a residential connection.  

One might ask "What's the point?! If you're going to be using 3rd parties anyway, you could always just use Gmail or Yahoo!".  

The main point here is that all of this can be done using your own domain, and all of the mail is stored on your server.  You control your destiny.  You can shange providers at any time if you like.  

You can have unlimited mail addresses, and unlimited aliases as well.  For example, instead of using something like SimpleLogin or addy.io, you can simply add aliases to CloudFlare, or on the MailPlus side.  

But wait, there's more!

One particularly interesting thing I recently found is projects like AgenticEmail, CloudMail and MailFlare that run on CloudFlare workers and allow you to process, store, and view incoming mail, without a specific server, and for free.  

Sure, Gmail is free, but they also harvest your data.  With cloudflare, you aren't rending an complete email service from them, so much as a generic web application and data storage platform.  Their free tier is generous, and their technology is top notch.  

Since you are renting the infractructore only, they don't control your mail - you do.  It's basically like renting a virtual server in a data center and then setting up a mail server - except that it's a serverless architecture.  

So you can (without having your own server) set up your own reliable GMail type system with relative ease.  

With CloudMail, for example, you can set it up so that all received messages are also forwarded to an outside address.  This lets you configure it such that all messages are received by CloudMail on CloudFlare, and then forwarded to your private on-premise mail server as well.  

You can always disable the forwarding, or switch to forwarding only - but having mail in two places serves as a redundant backup.  For example, you can set CloudMail to retain only 30 days worth of email for quick access from anywhere, but keep everyone on your local server where you have virtually unlimited storage.  

With this hybrid approach, you can have all of the control of on-premise hosting combined with all of the benefits of cloud based web mail - all while avoiding many of the drawbacks of both.