Dovecot problem receiving email from ISP IMAP mail account

[spaceuser]

Fetchmail seems to see how many mails to download, but Dovecot answers with error in /var/log/mail.log. Don't understand. Reading this forum on Dovecot points on problems with certificate, but I'm not sure where to start looking for trouble.

Log:

Code: Select all

Sep 24 11:41:38 b3 dovecot: IMAP(lars): Disconnected: Logged out bytes=840/2681
Sep 24 11:41:38 b3 dovecot: IMAP(lars): Disconnected: Logged out bytes=160/576
Sep 24 11:41:48 b3 dovecot: imap-login: Disconnected (no auth attempts): rip=192.168.10.55, lip=xyz.xyz.93.87, TLS handshaking: SSL_accept() failed: error:14094418:SSL routines:SSL3_READ_BYTES:tlsv1 alert unknown ca
Sep 24 11:42:06 b3 dovecot: imap-login: Login: user=<lars>, method=PLAIN, rip=192.168.10.55, lip=xyz.xyz.93.87, TLS
Sep 24 11:43:11 b3 fetchmail[22305]: starting fetchmail 6.3.17 daemon
Sep 24 11:43:12 b3 fetchmail[22305]: 109 messages (109 seen) for user_x@telia.com at mailin.telia.com.

My Thunderbird client can access to B3 user mail account via IMAP on port 993, from both internal as external network, but only reading B3 local mail. I have before trying to retrieving mail to B3 from my ISP, using IMAP over port 993 to my ISP directly, with out any problem (using Thunderbird). Same account information is added on B3 web interface for user/email settings. Tried both mailin.telia.com:993 and without the port number. Since fetchmail could see how many mail there are in ISP mailbox, I can't see there should be a connection problem to my ISP from B3. Please correct me if I'm wrong here.

Tried to login using Horde webmail client, and I could see same B3 local mail as Thunderbird. Both clients could send email (using ISP SMTP).

Any suggestions on where to start checking would be appreciated.

TIA

[Ubi]

i find it difficult to understand what itis you are trying to do, or what you consider to be the error that dovecot throws. In any case, dovecot will not retreive mails from a remote server, only dish up local mails. You can use fetchmail to inject remote mails into the local queue, but afaik dovecot has very little to do with that process.

[spaceuser]

Sep 24 11:43:12 b3 fetchmail[22305]: 109 messages (109 seen) for user_x@telia.com at mailin.telia.com

As seen above, fetchmail seems to find 109 emails within the email account at my ISP, telia.com. I was thinking that if fetchmail can't put the email into Dovecot for some reason, since Dovecot also presents an error in logfile.

To clarify what I'm going to do. Today I have all my email at my ISP. I normally read my email using Thunderbird email client, on Linux Mint at home, and on Windows 7, at work, and on my HTC Desire mobile as well. All client uses secure IMAP, (port 993) to read email from my ISP. Everything is working fine, but the quota on my ISP is nearly full. Therefore and since I've bought my B3 server this summer, my plans where to configure the B3 so it will fetch all email from my ISP. After that I could change my email clients to check my B3 instead of my ISP, still using IMAP, since I like to have all emails on one place. My plan was that fetchmail should rectrieve all email from my ISP account, so I don't need to worry about any quota.

The first thing I did to reach that goal, was to setup an account on B3. I followed the instructions from the B3 manual http://download.excito.net/web/B3/B3-UM-eng.pdf on page 84-86. I reentered all email account id and password, to be sure that wasn't the issue, and as the last line from the log says: 109 messages /109 seen).

Hope that clarifies what I'm doing and what I like to do. I did expect that it should be enough to configure the email using the B3 web interface, but since it's not working and the error messages, I wonder.

[Ubi]

On page 84-86 there's a description on retreival of local mail. Fetchmail is described on page 82-83. For now I take it you also followed those instructions.
* are you sure the fetchmail server is running?
* are you sure the smtp (send and receive) server is running?

I tried my gmail account with the following settings
* server: imap.gmail.com
* protocol: imap
* use encryption: yes
* leave mail on server: yes

Do you have a gmail account (or can you make one) so you can test with these same settings?

hope this helps
ubi

[spaceuser]

You're right, fetchmail is described on page 82-83. My copy of the manual was created in dec 02, 2010 and modified januari 27, 2011. So, I downloaded the current version of the manual again (i download my copy when I bought the B3 in july). Anyway... I have followed the instructions for fetchmail in the manual.

I have setup my gmail account as a second account on my B3. No email is retrieved from gmail, but one other very strange thing have happend. The B3 have retreived one new email from my ISP account, but only one, but there are a large number of email there, and probably around 109 email, as said before, in the inbox. So I'm asking myself why suddenly will the server retreive just only one email. There's been 30 minutes since I added the gmail account, and I know for sure there are emails to retreive in there.

The /var/log/mail.log file says that 37 email could be seen in gmail account, but no email is retreived. My ISP account now says there are 110 emails to be seen. 109 + the new one, since I have the the "leave mail on server" activated at the moment.

* are you sure the fetchmail server is running?
* are you sure the smtp (send and receive) server is running?

I tried a top command, but didn't find fetchmail in the listing, so I have restarted the fetchmail, but that didn't change anything. If the smtp server is running, I could not say. How can I check?

LATER:
Even one more email has come, and now from my gmail account. It seems that only new emails are retreived from my ISP as well as from gmail.

[Ubi]

you can check if the servers are running from the web GUI of the B3, if you log in as admin. Go to the services tab.

[spaceuser]

you can check if the servers are running from the web GUI of the B3, if you log in as admin. Go to the services tab

Yes, I forgot that. I did check that, but all email services was already activated.

Any suggestions on why only new email seems to be retrieved by the B3, and not all of them?

[Ubi]

the answer is in /var/log/mail.log, but your problem is not with dovecot, but with fetchmail and postfix. In my case, I did not enter the hostname of the B3 correctly and the postfix server bounced all mails meant for the B3 local queue back onto the original sender. This was all in /var/log/mail.log though.

[spaceuser]

I have used Linux for a number of years, but still I feel myself as a newbie. Therefore it seems to me that you need to be a real expert to solve any problem, that in the first place should be so easy to setup, with the B3 web interface and all.

the answer is in /var/log/mail.log, but your problem is not with dovecot, but with fetchmail and postfix.

I can live with that it's not Dovecot that are the main reason for all my problem, but I find the B3 system very sensitive to any change you make. In my case, I feel the main problem started when I changed the setup from "Router + Firewall + Server" profile to "Server only" profile for a short time, and my move back to the "Router + Firewall + Server" profile.

About the /var/log/mail.log file, it says almost nothing to me about the reason for why it only will retreive new email, and not all of them.

Code: Select all

Sep 25 08:00:04 b3 postfix/pickup[31789]: ABB8624002: uid=0 from=<root>
Sep 25 08:00:04 b3 postfix/cleanup[31943]: ABB8624002: message-id=<20110925060004.ABB8624002@b3.localdomain>
Sep 25 08:00:04 b3 postfix/qmgr[23403]: ABB8624002: from=<root@b3.localdomain>, size=831, nrcpt=1 (queue active)
Sep 25 08:00:04 b3 postfix/local[31945]: ABB8624002: to=<root@b3.localdomain>, orig_to=<root>, relay=local, delay=0.14, delays=0.07/0.03/0/0.03, dsn=2.0.0, status=sent (delivered to maildir)
Sep 25 08:00:04 b3 postfix/qmgr[23403]: ABB8624002: removed
Sep 25 08:01:21 b3 fetchmail[26359]: 111 messages (111 seen) for user_x@telia.com at mailin.telia.com.
Sep 25 08:01:22 b3 fetchmail[26359]: 38 messages (38 seen) for user_y@gmail.com at imap.gmail.com.
Sep 25 08:06:23 b3 fetchmail[26359]: 111 messages (111 seen) for user_x@telia.com at mailin.telia.com.
Sep 25 08:06:24 b3 fetchmail[26359]: 38 messages (38 seen) for user_y@gmail.com at imap.gmail.com.

As seen, no error messages here.

I have now tried to add my domain to the email settings in the web interface, according to the B3 manual, page 84, but even after a restart, nothing have changed. I have no more ideas to where to look or what to do. I don't what to mess up the system even more, if it's possible to "rescue" it as it is. If anyone have any tip or hint on any config file orlog file content I should be looking for, or anything. Please let me know.

LATER:
Did find a "fetchall" parameter I added to the /etc/fetchmailrc file for my gmail account just to try, and it have now download all email from my gmail account. SInce I had the "keep" parameter, I was warned that it's not a useful setup, since it will download duplicates of all email for every session, but this was only a test. Anyway, it seems to work, and that is a real step in the right direction. I will test further on here.

[Ubi]

if the fetchall works, it kinda looks like fetchmail thinks it already has all the messages. This points to a problem in the fetchmail cache file or something writing to this file. Your fetchmail log confirms this (111 messages (111 seen), meaning 111 were already considered seen = previously downloaded)
I ran an strace on fetchmail and it seems to go for a file called ~/.fetchids. Do you have this file?
ALso, is the time correct on your B3?
So in short, your fetchmail thinks it has already downloaded the old messages, and therefore will only download the new arrivals. Running fetchmail once with a fetchall flag willl download your old stuff, and, if you remove the flag, it will download the newly arrived messaged.
To test this, you need to send a mail to your fetched address and see if that mail gets downloaded by fetchmail.

[spaceuser]

I ran an strace on fetchmail and it seems to go for a file called ~/.fetchids. Do you have this file?
Also, is the time correct on your B3?

No that file I couldn't find. Do you think I absolutly need it? The date/time is correct. No problems there.

I have now set the fetchall parameter in /etc/fetchmailrc file, and all 111 emails where downloaded from the inbox folder. After that I remove the parameter. I have a number of other folders created on my ISP email account and these folders where not downloaded at all. Since I have created a "test setup" in my Thunderbird email client, I could easily mark the folders in my current setup in Thunderbird and "Copy To..." the "test setup". So for the moment I find myself quite pleased with the result. Before I remove the "keep" parameter in the /etc/fetchmailrc file, I will run as it is for a couple of days on all my email clients to see that all runs as it should.

Ubi, thanks a lot for all your input these days!

[Gordon]

Choosing this topic to express my annoyance (and possible solution for the topic starter).

I never got around to setting up the mail server before because both my former and my current ISP is blocking port 25, but last week I saw mentioning of a secondary route through one of my ISP's servers; it's a bit of a hack but it actually works: yeah!

So now that I was finally able to receive mail I obviously wanted to be able to read it as well, causing me to revive this topic because I couldn't To start with I could not log in to Horde, which I found to be caused by Dovecot running but doing absolutely nothing and I had to manually edit the dovecot.conf file to make it start imap(s) and trip onto the next problem. Although I could now login to Horde, it still showed my inbox as empty; so where did my mail go? The solution is as simple as it is stunning: Dovecot is not looking in the same place as Postfix is configured to store the incoming mail. You guys should really fix this this for new buyers.

Here's the two sections in dovecot.conf that you should change:

Code: Select all

#This setting is pointless:
protocols = none

#To make any sense (also with firewall configuration options) this should read
protocols = imap imaps

Code: Select all

#So what is the default?
#mail_location =

#Since mail is not showing in Horde we can safely assume it's not were Postfix puts it: 
mail_location = maildir:~/Mail

[johannes]

@Gordon, this works for me so this has to be a new issue (or that you meesed it up yourself, but I assume this can't be the case).

What sw version are you on so I can try to reproduce?

[oliver]

@Gordon: I had no stress with my installation (B3 as server + router ....) and Horde. The emails came to Horde without changing anything...The only bad thing was that the web interface did not accept "#" and some other special signs in your email passwords I told johannes already and they accepted that as bug which will be fixed within the next version ...

Take care

Oliver

[Gordon]

@Gordon, this works for me so this has to be a new issue (or that you meesed it up yourself, but I assume this can't be the case).

What sw version are you on so I can try to reproduce?

Hi Johannes,

Running version is 2.3.2, but that is not the original install version. From what I gather, conf files are rarely replaced so the issue may have been in that (unknown) version.

I'm actually somewhat puzzled as I seem to remember that I did have a quick glimpse at Horde when the B3 came in, so it must have worked then. Also, when I created a new user today I noticed that the "Mail" folder was not automatically added, but got created when I used that login with Horde (that was after I had rectified the incorrect entries in dovecot.conf). Since "Mail" did exist in my own account that must mean that my memory is correct.