MikeL's FreeBSD howto - popper and imap
[20240322]
Today I'm working on getting my email secondary up-to-speed. I was getting "no such user" for everybody incoming. This includes root, which caused sendmail panic, cannot save anywhere. Turns out, it was just bitching because I hadn't updated my local-host-names file. It should have been sufficient, it looked like the one I've been using on the master up until recently, but replacing it with the new one I've made for dkim-trusted hosts seems to have fixed it.
[20240228]
BTW: The sasl username/password pairs are stored in a mixed file, but can be observed in plaintext (with appropriate permissions of course). The file is at:
/usr/local/etc/sasldb2.db
In theory, moving this file to another system should bring the old system's users passwords in. If you're going to do this, probably a good idea to first:
service saslauthd stop
[20240226]
Did some googling on how to set up cyrus to handle the appropriate certificate for each of different domains. It's not really that hard, but the result is basically that you have a set of processes running, each handling it's own domain. The heck with that, I'll just tell both my users to accept my main hosting certificate in their email reader.
[20240218]
All right, so in essence, to move imap and data from an old 32bit server to a new 64bit:
[20240218]
Was getting an error about lmtp socket - fixed with:
cd /var/imap
mkdir lmtp
chown cyrus lmtp (should be 755 cyrus:cyrus)
service imap restart

To just start over with a blank slate, rename your test dir out, and make the new dir and socket dirs.
service imap stop
cd /var
mv imap imap.bak
mkdir imap
chown cyrus imap (should be 755 cyrus:cyrus)
cd imap
mkdir socket
chown cyrus socket (should be 755 cyrus:cyrus)
and make the lmtp socket dir as above.
service imap start

Doing a bunch of googling, I kept running into commands like 'reconstruct' and the examples always showed 'user.name' using a dot. As my console log is showing the same, I assumed this was correct. No - I presume this is because I have 'unixhieracrhysep=yes in my config file. Doing:
reconstruct -r -f user.mike
gave no error, and nothing on console. However, same command with a slash '/', gave a bunch of console errors and fixed the error, allowing thurnderbird to get email instead of giving a "no such user" error.

Now trying to get the mb data from the old 32 bit server. Looks like it's all stored in /var/spool/imap, so am 'tar'ing that up to bring it across.

[20231126]
I'm in the process of changing over to Starlink, and learned that they intercept port 25. In your email sender program you will need to use port 587, not port 25. (This really is the proper configuration, port 25 is meant for server-to-server SMTP, not client to server.) If you're trying to do server to server on port 25, you're simply out of luck.
P.S. I gave up on Starlink, throughput-wise it was no better than my existing Centurylink DSL. It's crazy-expensive to get fixed IP addresses, and I have servers that need port 25 access.
[20230116]
Yesterday's hanging after connect was really simple, if I had just paid attention. This morning I look at /var/log/console.log, and looky that, there's imap and pop3 errors! I guess I wasn't thinking real clearly yesterday to not check the logs. Also of course, check /var/log/imapd.deb.
Jan 15 21:02:28 pedicel kernel: Jan 15 21:02:28 pedicel pop3[9677]: [pop3d] error initializing TLS
Jan 15 21:02:28 pedicel kernel: Jan 15 21:02:28 pedicel imaps[9679]: unable to get certificate from '[path omitted]/SECURE.VINTNERS.NET.crt'
Jan 15 21:02:28 pedicel kernel: Jan 15 21:02:28 pedicel imaps[9679]: TLS server engine: cannot load cert/key data, may be a cert/key mismatch?
This was an old dead certificate, long ago replaced in Apache - I guess when I installed it I forgot the change it imapd.

When I changed this over to the new cert, I got mildly useful log errors similar to the above. It helpfully says "may be a cert/key mismatch?" which is totally misleading, it was actually a simple unix filesystem permissions problem. To determine this, I temporarily made the cert and key file world read. Everything started working, yay! So I then changed it to group readable, and added cyrus user to mail in /etc/group. Note! At this writing, this does not fix it - I have no explanation why adding cyrus to mail group does not work - at this time the damn key file remains world-read as I don't have time to mess with it further.

Oh, and to find out what user a daemon is running as, do ps ax -u.

[20230115]
I've been beating my head on the wall for this forever, it's not obvious, so I'm gonna write about it.

I have cyrus/sasl installed and running. I am set up in Thunderbird and am able to access mailboxes, but ONLY VIA POP3. I have not configured a cyrus account, or created the individual accounts. I have added the accounts with saslpasswd2. This is enough for POP3, but is not enough for IMAP. The symptoms are that when trying to set up an imap mail account, Thunderbird will pop up a notification something to the effect of "no such mailbox". Yet POP3 works just fine.

More googling.

Define password for cyrus account:
saslpasswd2 cyrus
[new passwd twice]
While you're at it, define password for additional accounts using same method as above.
Create cyrus admin account:
cyradm -u cyrus [your server]
You may start bumping into permissions problems - note that these are not standard unix file permissions - here's a helpful page: lrswipkxtecda in Cyrus at serverfault.com.
Create that users mailbox in IMAP world:
createmailbox user/[user]
listmailbox
Add any other users using same method.
quit
Doing this above, simply gave me a "permission denied" error. I fixed this by going back into /usr/local/etc/imapd.conf and commenting in the "admins" line and giving the value "cyrus".
Start rant here: I'm still really annoyed at how much extra crap has to be done for each user. I should not have to add each user to sasl for access, and I should not have to add them each to imap. I thought I had configured sasl to work directly off the users in passwd - evidently not. End rant.

[20221228]
New install of imap. Getting OTP unavailable because can't read/write key database /etc/opiekeys: permission denied
I tried setting the two /etc/opie files to 777 - same error, so I know it's not really a permissions problem. Let's rebuild /usr/ports/security/cyrus-sasl2. Do a make config and deselect the OTP option. You may have to monkey around with make deinstall, then make install. This seems to have fixed this problem.

[20221209]
In maillog I've been seeing:
    sm-mta[88243]: STARTTLS=client: file [path hidden]/ov_chain.txt unsafe: Permission denied
I've tried every stupid group, I eventually made the thing completely world read. Still get the error. Finally figured it out. This is a case where sendmail is lying to you. This is not the actual path being handed to the file open call. This is the path called out in the config file, which is then concatenated to another config parm. Specifically:
In sendmail.mc:
define(`CERT_DIR', `[path hidden]')
define(`confCACERT_PATH', `CERT_DIR')

dnl WRONG!
define(`confCACERT', `CACERT_DIR/ov_chain.txt')

dnl CORRECT!
define(`confCACERT', `ov_chain.txt')

Apparently, sendmail concats CACERT_DIR onto CACERT_PATH to create a fullpath to the file. When there is an error, it does not display the actual path it was trying to open (which would have made the problem immediately obvious), but instead simply displays the value of one of the two parms. LAME!

Oh, and lastly, the cert dir permissions are owner=root, group=mail, world=no access. I had added mailnull and others to group mail, but it seems those are unnecessary. (See above 2023016 - you do want to add "cyrus" to the "mail" group.

[20211114]
Been getting a log full of the following:
imap[3717]: IDLE: error sending message INIT to idled for mailbox user.xxxxxxxxx: No such file or directory. Falling back to polling every 60 seconds
This was fixed by ensuring that idled was running. Check that /usr/local/etc/cyrus.conf has the line:
idled cmd="idled"
in the DAEMON section.
Note that once I had enabled this, I tried doing a service saslauthd restart, but it did not start the idled daemon. I then just rebooted the server and it did start up, so I am not sure who needed to be restarted to make this happen.

Been having a helluva time getting customers to be able to send through my server. Finally got it working, here's some of the details, who knows which parts are needed.
saslpasswd2 [customer-username@my-hosting-domain]
Just setting pswd for their username alone is not sufficient, you'll simply get bad password errors when trying to send.
Their Thunderbird SMTP settings should be:
Server Name: smtp.[customer-domain]
Port: 587
Connection security: STARTTLS
Authentication method: Normal password
User Name: [customer-username]
No domain on username.

20200501
Each morning in my console log email, I'm seeing a bunch of the following:
kernel: sonewconn: pcb 0x275fad94: Listen queue overflow: 16 already in queue awaiting acceptance (2 occurrences)
Over the past few months, whenever I've had time to waste, I've done google serches and spent many hours wading through junk, and I think I have the solution. It's actually extremely simple, you just have to find it...

In your /etc/rc.conf, add:
cyrus_imap_flags="-l 64"

I spent a lot of time dorking around with:
sysctl kern.ipc.somaxconn
which many posters said will fix it - they thought the problem was that the system max was being exceeded. (I never did get this to stick when I added it to /boot/loader.conf -- the above command always showed the default of 128.) I tried setting it on the running system with:
sysctl kern.ipc.somaxconn=256
which I could then verify with the above command, but the next day I still had new errors in the log.
In my searches, I found a page talking about the 'sonewconn' warning, and it had a terse reference to "in the performance settings I found you can add "-d -l 128" (dash lower L not dash one). It didn't specify where, but I guessed it might mean in rc.conf. Doing a ps ax | grep cyrus I determined that the actual executable is 'master'.

My cyrus imap install did not install the man pages for cyrus/imap, and the Cyrus Team documentation web pages didn't show me either. You need the command line switches for the 'master' program. You'll find this file already on your system at:
/usr/local/share/doc/cyrus-imapd/text/imap/reference/manpages/systemcommands/master.txt
(Once I knew exactly what to look for, "cyrus listen queue backlog", it is present at the Cyrus Team webpage.)
The default for value "-l" is 32, so I've just arbitrarily doubled it. I'll try to remember to get back here in a few days and report if that took care of it.
[20200504] Doesn't seem to have fixed it.

Oh, and just a little background - my system has but a couple dozen users, all very low use. These customers are all local small businesses, no international issues. So I know these guys are not causing so much activity that they're overflowing socket buffers.
The problem is that I'm being constantly pounded by dictionary attacks -- about 8,000 failed email login attempts per day. About 80% are from 3-4 IP addresses, and use dozens of rotating completely bogus usernames; about 20% are for a single username but from from differing IP addresses; and a handful of other attempts.
I've manually added the top 10 bad guys to /etc/mail/access as rejects, but it hasn't made any difference to them, they're not checking their error return and giving up even though they're now getting "access denied" instead of "unknow user/pswd". In the future I'll get around to getting 'ipfw' working again, and add 'fail2ban'.
20200309
Was getting the following log error:
Mar 9 11:04:11 rachis sm-mta[59109]: STARTTLS=client: file [path omitted for security].key unsafe: Group readable file
Had to add the following to /etc/mail/[your-system-name].mc
define(`confDONT_BLAME_SENDMAIL', ``GroupReadableKeyFile'')
And of course sendmail-restart
Ref here

BTW: We need the file group-readable as we are sharing it with imap/cyrus and with Apache.

20200307
I seem to have everything working now...

I was getting the following in console.log
Mar 7 11:59:19 rachis kernel: Mar 7 11:59:19 rachis imaps[2696]: imaps: required OpenSSL options not present
Mar 7 11:59:19 rachis kernel: Mar 7 11:59:19 rachis imaps[2696]: Fatal error: imaps: required OpenSSL options not present
Mar 7 11:59:19 rachis kernel: Mar 7 11:59:19 rachis master[2411]: process type:SERVICE name:imaps path:/usr/local/cyrus/libexec/imapd age:0.063s pid:2696 exited, status 75
This is expected, as I have imaps and pop3s enabled. Now it's time to actually get TLS working...
I set the following in /usr/local/etc/imapd.conf
tls_server_cert: [same as .mc file confSERVER_CERT]
tls_server_key: [same as .mc file confSERVER_KEY]
tls_server_ca_file: [same as .mc file confCACERT]
tls_server_ca_dir: [same as .mc file confCACERT_PATH]

and service imapd restart, now in console.log log I'm getting:
unable to get certificate from '[path omitted for security]'
unable to get private key from '[path omitted for security]'
I double-checked my pathnames, they were correct. I checked the directory permissions for the certs, they're fairly tight - this is enforced by sendmail. Just to find out if this could be the problem, I noted the file permissions on the cert file and cert key file (they were 640), and changed to 644 (world read), then did restart. Now no error in log, and client allows me to set TLS, yay! So the problem is indeed file permissions. chmod back to 640. To fix this I just had to add user cyrus to group mail.
Edit /etc/group
mail:*:6:
becomes:
mail:*:6:cyrus
And in the cert dir:
chgrp mail [the 4 files called out above in imapd.conf]
Note that it did not bitch when I neglected to make the 'ca file' accessible, so this may not be used - I've chgrp'ed it too in order to avoid possible unseen future problems.

20200228
Installed a whole new server. Now doing imap/pop3 for my customers, using imapd.

pkg install cyrus-imap30
Edit /usr/local/etc/cyrus.conf

(References:)
https://github.com/cyrusimap/cyrus-imapd/issues/2171
https://github.com/cyrusimap/cyrus-sasl/tree/master/pwcheck
In /usr/local/etc/imapd.conf, leave most stuff as default, make following changes: As directed by the package install, run:
    /usr/local/cyrus/sbin/mkimap

Add to your .mc file
  define(`confLOCAL_MAILER', `cyrusv2')
  MAILER(`cyrusv2')
Rebuild .cf/restart sendmail.

Start the server:
    service imapd start
20190130 - more on qpopper logspam.
The comment below (20181008) about "openio_verbosity doesn't work. However...

By adding it, there's now a message in the log, that indicates this is not a valid option, and proceeds to list all the valid options, yay!
But again, of course, none of the remaining options seem to be related to my current problems, namely:
    Client address "80.82.65.82" not listed for its host name "no-reverse-dns-configured.com"
and:
    support@vitners.net at 94.102.53.10 (94.102.53.10): -ERR [AUTH] Password supplied for "support@vitners.net" is incorrect.
Both of these errors are of absolutely no use to me, in both cases it's just another bullshit dictionary attack that I can do nothing about - I simply don't even want to hear about it.

[20181008]

Skip popper, use imap. Yeah, I know, some old customers are too difficult to move to imap...

Sorry, I did not document my original install of popper (I used qpopper for some reason, now lost to time), and imap.

I'm getting logspam from qpopper and imap. Starting to document the attempts here to reduce the problem.

If you don't have it already, you may want to find a sample imapd.conf file from the ports tree, and make changes, or simply add the following to a new file:
    openio_verbosity=quiet
I don't know yet if this works, but I wanted to document so I wouldn't forget it.

[20181206]

qpopper is incredibly noisy, contantly jamming your syslog full of logspam. The problem with this is that it's basically a silent DOS attack. Each day I get emails from the system to alert me of problems, but I don't see real problems due to the tens of thousands of lines of bogus email login attempts. There's nothing I can do about it, there's creeps all over the planet constantly trying to get in - that's just the way things are. Yes, I should be worried, but the only thing I could do would be to block by IP address, which is constantly changing. Nope - just disable these error messages so you can see the rest of the log.

Create a /etc/mail/qpopper.conf

#
# 20181206 MikeL
# config file for qpopper

set debug = false
set statistics = false
set reverse-lookup = false
set log-facility = mail
Note that the first 3 lines get rid of specific errors, the last line simply says to put qpopper output into /var/log/maillog, which is already so obscenely noisy that a little bit more won't matter.
Sample msgs eliminated (this is to aid search engines to find this page):
    qpopper[30790]: zzzz at zzz.zzz.zzz.zzz (zzz.zzz.zzz.zzz): -ERR [SYS/TEMP] POP authentication DB not available (user zzzz): No such file or directory (2)
Note that this one is coming with the login of a legit customer who has never complained about not getting email, so from everything I can see, this error is utterly bullshit -- completely benign. I was unable to eliminate this msg, the answer for this was the last line which moves it to the maillog file.

    Stats: zzz 2 5731 0 0 zzzz.domain.com zzz.zzz.zzz.zzz
This is eliminated by the statistics line.

I'm not sure if debug=false is helping.

Note that if you're running popper under inetd, you need simply edit the config file. It is read each time an email comes in as it's a fresh program startup. Otherwise, of course, you'll need to do a popper restart.

[old, old, old, ignore]
This is basically a checklist of how to make popper work. The intent is not to cover everything in detail, merely to point you to each of the tasks that must be performed.

Popper is the program that allows client email programs to fetch their email from this server -- the old way; via POP3. I've not messed with IMAP, so don't even ask.

First time only

Copyright © 1995-2024 Mike Lempriere (running on host bayanus)