The Eternal Sync Bug

This one of of the most reported bugs in all of Mailspring. Under many circumstances, client syncing takes an extremely long time, approaching eternity. It also tends to use excessive CPU power in the process.

I’m collecting all the reports about this here. There’s too much information to migrate. See the individual reports for more information.


Possible Related:

Hi,

First time ever posting a bug, so my apologies if things are missing or are unclear.

Problem

I tried using Mailspring for several months now, most mail accounts are syncing fine. However iCloud keeps syncing. I can use it, but in the left bottom it keeps saying ‘Scanning messsages…’. At initial install, nearly all dots of iCloud account under scanning messages were green, except for ‘Inbox scanning’, this stalls at 71%. After restart it is not doing anything anymore, except scanning (and taking recources trying to scan).

Context

No

Email Provider

iCloud mail. Other seem to be working fine. However iCloud is my ‘main’ email.

Setup

Arch Linux 64 bit
Mailspring 1.7.8-13635bcf

I tried opening mailspring logs, i only see this:

1430 [2021-01-19 20:38:15.530] [foreground] [info] Idle exited with code 0
1430 [2021-01-19 20:38:15.553] [foreground] [info] syncFolderChangesViaCondstore - INBOX: modseq 609 to 609, uidnext 255 to 255
1430 [2021-01-19 20:38:15.560] [foreground] [info] Idling on folder INBOX
1430 [2021-01-19 20:39:16.629] [background] [info] Syncing folder list...
1430 [2021-01-19 20:39:16.667] [background] [info] syncFolderChangesViaCondstore - INBOX: modseq 609 to 609, uidnext 255 to 255
1430 [2021-01-19 20:39:16.693] [background] [info] syncFolderChangesViaCondstore - Sent: modseq 90 to 90, uidnext 67 to 67
1430 [2021-01-19 20:39:16.709] [background] [info] syncFolderChangesViaCondstore - Drafts: modseq 239 to 239, uidnext 81 to 81
1430 [2021-01-19 20:39:16.724] [background] [info] syncFolderChangesViaCondstore - Archive: modseq 7 to 7, uidnext 9 to 9
1430 [2021-01-19 20:39:16.742] [background] [info] syncFolderChangesViaCondstore - Deleted Messages: modseq 4 to 4, uidnext 3 to 3
1430 [2021-01-19 20:39:16.762] [background] [info] syncFolderChangesViaCondstore - Junk: modseq 1 to 1, uidnext 1 to 1
1430 [2021-01-19 20:39:16.782] [background] [info] syncFolderChangesViaCondstore - Mailspring: modseq 1 to 1, uidnext 1 to 1
1430 [2021-01-19 20:39:16.798] [background] [info] syncFolderChangesViaCondstore - Mailspring.Snoozed: modseq 1 to 1, uidnext 1 to 1
1430 [2021-01-19 20:39:16.814] [background] [info] syncFolderChangesViaCondstore - Templates: modseq 1 to 1, uidnext 1 to 1
1430 [2021-01-19 20:39:16.831] [background] [info] syncFolderChangesViaCondstore - Trash: modseq 123 to 123, uidnext 94 to 94
1430 [2021-01-19 20:39:16.845] [background] [info] syncFolderChangesViaCondstore - INBOX.spam: modseq 1 to 1, uidnext 1 to 1
1430 [2021-01-19 20:39:16.846] [background] [info] Sync loop deleting unlinked messages with phase 1.
1430 [2021-01-19 20:39:16.846] [background] [info] Sync loop complete.
1430 [2021-01-19 20:40:15.758] [foreground] [info] Idle exited with code 0
1430 [2021-01-19 20:40:15.775] [foreground] [info] syncFolderChangesViaCondstore - INBOX: modseq 609 to 609, uidnext 255 to 255
1430 [2021-01-19 20:40:15.779] [foreground] [info] Idling on folder INBOX
1430 [2021-01-19 20:41:16.846] [background] [info] Syncing folder list...
1430 [2021-01-19 20:41:16.884] [background] [info] syncFolderChangesViaCondstore - INBOX: modseq 609 to 609, uidnext 255 to 255
1430 [2021-01-19 20:41:16.900] [background] [info] syncFolderChangesViaCondstore - Sent: modseq 90 to 90, uidnext 67 to 67
1430 [2021-01-19 20:41:16.916] [background] [info] syncFolderChangesViaCondstore - Drafts: modseq 239 to 239, uidnext 81 to 81
1430 [2021-01-19 20:41:16.935] [background] [info] syncFolderChangesViaCondstore - Archive: modseq 7 to 7, uidnext 9 to 9
1430 [2021-01-19 20:41:16.951] [background] [info] syncFolderChangesViaCondstore - Deleted Messages: modseq 4 to 4, uidnext 3 to 3
1430 [2021-01-19 20:41:16.969] [background] [info] syncFolderChangesViaCondstore - Junk: modseq 1 to 1, uidnext 1 to 1
1430 [2021-01-19 20:41:16.985] [background] [info] syncFolderChangesViaCondstore - Mailspring: modseq 1 to 1, uidnext 1 to 1
1430 [2021-01-19 20:41:17.002] [background] [info] syncFolderChangesViaCondstore - Mailspring.Snoozed: modseq 1 to 1, uidnext 1 to 1
1430 [2021-01-19 20:41:17.019] [background] [info] syncFolderChangesViaCondstore - Templates: modseq 1 to 1, uidnext 1 to 1
1430 [2021-01-19 20:41:17.033] [background] [info] syncFolderChangesViaCondstore - Trash: modseq 123 to 123, uidnext 94 to 94
1430 [2021-01-19 20:41:17.052] [background] [info] syncFolderChangesViaCondstore - INBOX.spam: modseq 1 to 1, uidnext 1 to 1
1430 [2021-01-19 20:41:17.052] [background] [info] Sync loop deleting unlinked messages with phase 2.
1430 [2021-01-19 20:41:17.053] [background] [info] Sync loop complete.

I believe this is a duplicate of this: The Eternal Sync Bug

I think the same. Until the bug has been fixed, I switched to another mail client.

I also notice mails not showing up in my inbox, while the other mail client sees them (both imap). But I am not sure this is related?

It probably is related, and it’s an interesting clue.

Problem

When I mark any email as read or unread the mailspring starts “Syncing your mailbox…” with status “Marking as read” or “Marking as unread” and it lasts forever. In this case, the application heavily loads the processor. Closing-opening the app fixes the problem.

Context

I try to reproduce the issue with different Gmail accounts. It reproduces on accounts with thousands of emails but never happened on accounts with hundreds of emails.

Email Provider
Gmail

Setup

  • OS and Version: Ubuntu Linux 20.04
  • Installation Method: snapcraft and deb tested. The issue is on both
  • Mailspring Version: 1.9.1

That’s an interesting clue.

Syncing mailbox message shows up and stays. There is no cpu usage or network traffic going on. It looks like, all messages are loaded, not 100% since there are thousands of them. If I click on the Syncing … Message - It shows me “Deleting All messages in Trash …”

There are no messages in the trash folder anymore, but it still shows that message.

Screenshot 2021-06-26 at 15.39.13

System macOS 10.15.7
lates mailspring version 1.9.1

FYI. From time to time, some messages appear in the trash folder, which have not been deleted, nor can i delete them in the trash folder by clicking empty trash. They just show up there and will only disappear from the trash folder (and show up in the inbox like they should) if I go to accounts and click on rebuild cache. Is this related to the sync issue?

I think, if i delete 1 message from a thread, the whole thread shows up in the trash, instead of only the 1 deleted message. I could not reproduce it yet, but maybe have a look at the threaded view / resurface message feature.

Mail servers are my own, running on linux (CentOS) dovecot/postfix

Having the same issue here on iCloud

Other accounts (Gmail, IMAP) managed to sync fine. iCloud has significantly more emails though.

Resetting the cache starts the process over but it will stall at the same percentages after awhile. Mailsync log files don’t seem to give any information.

I have an interesting incidental finding, which may or may not be directly related to everlasting sync issues… I have disabled the “Show icon in menu / system tray”. My issue with this was Mailspring seemed to be constantly using at least 30 to 40% of one of my CPU cores once started, until I exited it from the tray icon. Now, the interesting bit is that by disabling that option and minimizing the client, the CPU cycles it consumes drops to 0% or almost 0%, while with the tray icon, it would keep consuming a lot of cycles even when no visible window (close or minimize), which leads me to believe there could be some sloppy code with a high rate polling being done by the tray icon. This is a wild guess, as I have not investigated the implementation, but could help finding the offending bit of code.

Email Provider
Gmail

Setup

  • OS and Version: Manjaro Linux x86_64
  • Installation Method: pamac via AUR build
  • Mailspring Version: 1.9.1-30ef802f

Using mailspring on macOS iMac - Catalina. When I delete a message using a gesture (swiping from left to right) - i get the eternal sync bug.

Screen Recording 2021-08-02 at 12.53.11

same issue with yahoo mail on arch linux. Stuck on “Scanning Messages”

Hello,

I think I have a very similar problem with a sync of the Office 365 email (Outlook) via IMAP on Windows 10 (with the latest client version).

There is an error in my log repeated multiple times (part from line 38 is repeating):
repeating_errors.log (2.4 KB)

Same issue here with an SMTP account :

16668 [2021-09-23 11:16:22.559] [background] [info] --sleeping
16668 [2021-09-23 11:18:22.559] [background] [info] Syncing folder list…
16668 [2021-09-23 11:18:27.126] [background] [info] syncFolderUIDRange for INBOX, UIDs: 41269 - 42019, Heavy: true
21706 [2021-09-23 11:19:09.516] [main] [info] Identity created at 1588348170 - using ID Schema 1
21706 [2021-09-23 11:19:09.518] [main] [info] ------------- Starting Sync (EMAIL) ---------------
21706 [2021-09-23 11:19:09.529] [metadata] [info] Metadata delta stream starting…
21706 [2021-09-23 11:19:12.527] [background] [info] Marking all folders as busy
21706 [2021-09-23 11:19:12.527] [background] [info] Syncing folder list…
21706 [2021-09-23 11:19:15.135] [background] [info] Syncing folder list…
21706 [2021-09-23 11:19:17.406] [foreground] [info] Connection does not support idling. Locking until more to do…
21706 [2021-09-23 11:19:18.974] [background] [info] syncFolderUIDRange for INBOX, UIDs: 41269 - 42019, Heavy: true
21706 [2021-09-23 11:19:24.529] [metadataExpiration] [info] Scanning for expired metadata
21706 [2021-09-23 11:19:24.530] [metadataExpiration] [info] – Will wake for next expiration in 7200sec
21706 [2021-09-23 11:20:36.385] [background] [critical]


*** Mailspring Sync
*** An exception occurred during program execution:
*** {“debuginfo”:“syncFolderUIDRange - fetchMessagesByUID”,“key”:“ErrorParse”,“retryable”:true,“what”:“std::exception”}


21706 [2021-09-23 11:20:36.385] [background] [critical] *** Stack trace (line numbers are approximate):
*** ??:? SyncWorker::syncFolderUIDRange(Folder&, mailcore::Range, bool, vector*)
*** ??:? SyncWorker::syncNow()
*** ??:? runBackgroundSyncWorker()
*** main.cpp:? main::{lambda()#3}::operator()() const
*** main.cpp:? _Bind_simple::operator()()
*** main.cpp:? thread::_Impl::_M_run()
*** thread.o:? execute_native_thread_routine()

Same here - I’m getting an error trying to sync my iCloud account and it fails in the same place:

4915 [2021-10-06 09:44:32.750] [background] [info] syncFolderUIDRange for Archive, UIDs: 1 - 4026, Heavy: true
4915 [2021-10-06 09:44:34.095] [background] [critical] 
***
*** Mailspring Sync 
*** An exception occurred during program execution: 
*** {"debuginfo":"syncFolderUIDRange - fetchMessagesByUID","key":"ErrorParse","retryable":true,"what":"std::exception"}
***

4915 [2021-10-06 09:44:34.095] [background] [critical] *** Stack trace (line numbers are approximate):
*** in mailsync     SyncWorker::syncFolderUIDRange(Folder&, mailcore::Range, bool, vector*)
*** in mailsync     SyncWorker::syncNow()
*** in mailsync     runBackgroundSyncWorker()
*** in mailsync     main::$_6::operator()() const
*** in mailsync     void* __thread_proxy(void*)
*** 0x7fff20392443  thread_start()
***

I’m stuck at the initial sync. It’s been sitting at scanning - inbox 2%, all the other folders 0%. It’s been like this for hours.

Email provider: Hotmail
System: macOS Catalina v10.15.7
Mailspring version: Version 1.9.2

Not really sure what I’m looking for in the logs (sorry new to this)… Seeing lots of this:

*** Mailspring Sync
*** An exception occurred during program execution:
*** {“debuginfo”:“syncFoldersAndLabels - fetchAllFolders”,“key”:“ErrorParse”,“retryable”:true,“what”:“std::exception”}


14841 [2021-11-12 10:41:47.268] [background] [critical] *** Stack trace (line numbers are approximate):
*** in mailsync SyncWorker::syncFoldersAndLabels()
*** in mailsync SyncWorker::syncNow()
*** in mailsync runBackgroundSyncWorker()
*** in mailsync main::$_6::operator()() const
*** in mailsync void* __thread_proxy(void*)
*** in mailsync thread_start()

Anyone know what to do?

1 Like

Same issue but with outlook/hotmail account for me.

But I have noticed that this is not problem only with Mailspring, but I am getting the same issue also with other email clients like Edison, Spark, Canary [all use MailCore2 !] even Apple mail (I am not sure if they also use MailCore)…

And it happens in multiple steps

  1. Issue with login account - getting the ErrorNamespace (IMAP)
  2. After multiple tries I get in but the sync doesn’t happen (endless sync) or is extremely slow.

The only app that doesn’t have issue for me is Microsoft Outlook.

Another observation is that this is happening only with my biggest account (over 15k email, existing for 10+ years)

----------IMAP----------
connect mailcore::IMAPSession:0x7ff7bfeff528

  • OK The Microsoft Exchange IMAP4 service is ready. [UABSADMAUAAyADUAMQBDAEEAMAAwADIANwAuAEUAVQBSAFAAMgA1ADEALgBQAFIATwBEAC4ATwBVAFQATABPAE8ASwAuAEMATwBNAA==]
    ssl connect imap-mail.outlook.com 993 2
    1 CAPABILITY
  • CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS ID UNSELECT CHILDREN IDLE NAMESPACE LITERAL+
    1 OK CAPABILITY completed.
    connect ok
    login
    2 LOGIN “ACCOUNT” “PASSWORD”
    2 OK LOGIN completed.
    3 CAPABILITY
  • CAPABILITY IMAP4 IMAP4rev1 AUTH=PLAIN AUTH=XOAUTH2 SASL-IR UIDPLUS MOVE ID UNSELECT CLIENTACCESSRULES CLIENTNETWORKPRESENCELOCATION BACKENDAUTHENTICATE CHILDREN IDLE NAMESPACE LITERAL+
    3 OK CAPABILITY completed.
    4 NAMESPACE
    4 BAD Request is throttled. Suggested Backoff Time: 47931 milliseconds
  • BYE Connection closed. 14
    fetch namespace failed

I think I know what causes this.

It’s the same problem I had when I used Thunderbird - my virus checking program continually kept scanning and rescanning the email directory on my computer. There is a page showing which directory to put as an “exception” in your virus scanning program (for Thunderbird) but I don’t know which directory I should list for MailSpring. One page that has the info (Thunderbird:Testing:Antivirus Related Performance Issues - MozillaWiki) and the directory that needs to be put into the exception: C:\Users[UserName]\AppData\Roaming\Thunderbird\Profiles.

Yes I know that is for Thunderbird (yada yada yada) but I’m 90-100% sure it’s the same problem with MailSpring.

Could someone PLEASE tell everyone which directory to list as an “exception” in virus checking programs please>

Thank you :slight_smile:
Luke

@lfrohling you are right :smiley: :raised_hands:. Thanks. In my case, I had problems connecting my custom IMAP account with an eternal sync but after I added an exception from my server It is working again. I really love this email client and it was frustrating not be able to use it.