Posted by: pre67vw - Tuesday, 21 February 2012 14:18:02
I've just gone through an upgrade to version 1.9.5.5
Initially the update had turned off everyone's notification emails :( , I then manually turned on my notifications and noticed that it updated the database yaf_user.NotificationType to 30 - so I then did an update query to set all users to this value. Notifications are now on, but everyone is receiving 3 notification emails instead of just one.
Can someone tell me where I'm going wrong, or point me to where I can fix this?
Thanks.
Posted by: JP - Tuesday, 21 February 2012 19:06:42
Have had this problem since 1.9.5. The record is 13 emails about the same thing...
On a lucky day one is sent, most common is 2-3 mails also for forum registrations and such...
Do not think it's an email provider thingy.... Rather a kind of too quick "oops, no contact - better try to send again" type of evaluation in the forum software... Didn't do that with 1.9,4 if I remember correctly.
Posted by: pre67vw - Tuesday, 21 February 2012 20:06:24
Oh dear...
Well I didn't have this problem in version 1.9.4 RC1 - maybe I can use the notification code from that version?
Posted by: JP - Tuesday, 21 February 2012 20:53:43
Thanks for confirming my notion. I started out with 1.9.4 RC1 and it worked well indeed.
The issue has been raised before in the forum, but was then "blamed" on ISP mail provider issues.
Maybe it's time to have a little look at the mail sending routine code to see if it needs an overhaul in logic?
Posted by: Dr-Hack - Tuesday, 21 February 2012 20:59:20
why is it when ever i come to the forum with something new that issue is just discussed already :p
so yes today for the first time i got dual mails of a topic i have watched .. so i guess it might happen with others as well .. :S ..will see the mail time and logs if anything matches
Posted by: pre67vw - Wednesday, 22 February 2012 11:58:40
It's going up... I just got 6 notification emails... [angry]
Posted by: pre67vw - Thursday, 23 February 2012 21:55:22
OK, well I think I've fixed my problem. Couldn't go back to 1.9.4RC1 because it looks like a lot of stuff has been re-written since then (Yaf.Core doesn't seem to exist in 1.9.4). So I downloaded the latest version and compared the notification code, changed a couple of lines and re-compiled. Now I'm getting one notification email and it all seems OK.
Posted by: JP - Saturday, 31 March 2012 22:36:15
Could you please publish the code changes?
My users are getting tired of getting multiple mails for each message posted in watched topics and such...
Posted by: tha_watcha - Sunday, 1 April 2012 08:06:45
[quote=JP;53972]Could you please publish the code changes?
My users are getting tired of getting multiple mails for each message posted in watched topics and such... [/quote]
This issue has been resolved in yaf 1.9.6.1 rtw, it was an issue with the task manager.
Posted by: pre67vw - Thursday, 12 April 2012 16:04:19
[quote=tha_watcha;53975]
This issue has been resolved in yaf 1.9.6.1 rtw, it was an issue with the task manager.[/quote]
Any chance you can be more specific? I'm on v1.9.5.5 and my hosting platform doesn't support .net v4
Thanks.
Posted by: Thantis - Thursday, 12 April 2012 19:50:32
Hmmm. I am currently using 1.9.6.1 rtw and I have now started getting 8 to 10 notice emails at a time also. Any help would be appreciated.
Posted by: squirrel - Thursday, 12 April 2012 20:35:28
I don't know how the code works, but I noticed on our forums (we're still running a patched RC of 1.9.5) - that if a mail gets stuck in the queue, what happens is if the queue doesn't complete, it keeps resending the messages that are successful -- when we flush out the queue, it runs fine until there's a problem with one of the messages or email addresses it'll get stuck in repeat.
Almost like it won't clear messages out of the queue unless it's able to complete the whole queue. We're going to go to 1.9.6 soon - I just haven't had the time to deal with it on our site yet --
I have noticed also that since we moved the forums to the new server, it's been sending notifications with no problems and no duplicates. Thera are big differences between the two servers: IIS versions, OS Versions, SQL Versions, and the change in mailserver. We don't use a pickup directory - we just talk to the IIS SMTP on an internal port and that's it. We used to talk to our public mailserver instead.
We did also recently go through the database and disable notifications for messages by default and instructed users how to go and turn them back on if they wanted them. The question is does the queue clear messages one by one if they send or does it process the queue as a 'batch' then clear them if the 'batch' is successful?
Just my notes --
Posted by: daveburke - Friday, 13 April 2012 00:33:36
[quote=squirrel;54130]
I have noticed also that since we moved the forums to the new server, it's been sending notifications with no problems and no duplicates. Thera are big differences between the two servers: IIS versions, OS Versions, SQL Versions, and the change in mailserver. We don't use a pickup directory - we just talk to the IIS SMTP on an internal port and that's it. We used to talk to our public mailserver instead.
[/quote]
Excellent summary, Squirrel. Thanks for sharing. I hadn't considered the local SMTP environment being an issue, but I have been running YAF on my dedicated server since 2008 and haven't had a single occurrence of multiple emails. And that was with YAF 1.9.3 through to 1.9.6.1. I shouldn't be tempting fate by publicly posting this, but I wanted to add my experience to the pool.
Thanks,
Dave
Posted by: francoz - Friday, 13 April 2012 00:38:52
I have faced multiple email problems with the newly installed 1.9.6.1 RTW version (on Windows 2008, SQL 2008 fully updated) which have suddenly disappeared when I have switched from my provider SMTP server (Postfix on Ubuntu) to one of our own servers which is not even in the LAN, but is reached over the Internet (mDaemon 8 on Windows 2003).
One thing I have noticed is that my provider server closes the SMTP connection because of time-out after a brief time (120 seconds), while our own server has got a very long time out setting, 15 minutes.
Posted by: pre67vw - Friday, 13 April 2012 16:00:22
I didn't have the problem a single time when I was on 1.9.4 RC1, now it's becoming a real issue.
It's on the same server with the same SMTP settings, the only thing that's changed is YAF :(
Posted by: pre67vw - Friday, 13 April 2012 16:23:28
Another issue that I have is that the notifications are in the language of the last poster on the thread. So not only do people get multiple emails, but they can't read them sometimes either...
Posted by: squirrel - Friday, 13 April 2012 16:34:57
[quote=pre67vw;54146]I didn't have the problem a single time when I was on 1.9.4 RC1, now it's becoming a real issue.
It's on the same server with the same SMTP settings, the only thing that's changed is YAF :(
[/quote]
Do you have any way to try a different MTA for your mail? setup a test account on another mail provider and set it in yaf and restart and see if the queue clears and duplicates stop?
That was one thing that changed in our configuration that sticks out where we're not getting 'duplicate emails anymore' since we moved our site --
Posted by: Thantis - Monday, 16 April 2012 01:56:12
So I might have found a solve for at least some of the people with this problem. Squirrel mentioned changing his users notifications so that they wouldn't receive them. I recently made changes to the database to change my user base's settings on notifications and I changed all rows to notify for all posts/forums. Thinking back on this, by doing so, I set the guest account to be notified in this manner also, but the guest account doesn't have an email address specified by default, which of course makes sense, but if Yaf is trying to send an email to a blank email address might this not be the cause? I have set an catch-all "no-reply@host.com" email address to the guest account and modified the notification for guest accounts back to the default. This seems to have corrected my problem. I'm sure this isn't the same problem everyone is having, but it might be worth taking a look into.
Posted by: squirrel - Monday, 16 April 2012 02:42:28
[quote=Thantis;54207]So I might have found a solve for at least some of the people with this problem. Squirrel mentioned changing his users notifications so that they wouldn't receive them. I recently made changes to the database to change my user base's settings on notifications and I changed all rows to notify for all posts/forums. Thinking back on this, by doing so, I set the guest account to be notified in this manner also, but the guest account doesn't have an email address specified by default, which of course makes sense, but if Yaf is trying to send an email to a blank email address might this not be the cause? I have set an catch-all "no-reply@host.com" email address to the guest account and modified the notification for guest accounts back to the default. This seems to have corrected my problem. I'm sure this isn't the same problem everyone is having, but it might be worth taking a look into.[/quote]
Which leads indirectly to if there is an email stuck in the queue and the queue doesn't complete, successful mails inside that same queue should be removed. I have to look at the source code and just how the queue is processed and cleared. It's an area I have no knowledge but would be nice to know how it works at this point.
Posted by: squirrel - Monday, 16 April 2012 04:48:07
Let me ask this of the devs:
The exceptions and errors responses I see in the source code show usage of the "DEBUG" flag. If I compile in debug mode and load it up, where will I find the 'debug' error messages? Do they go to the event log of YAF or are they on screen or what?
In the source code, it looks like all messages in the table are loaded into a dictionary object, spooled to the mailsender, then deleted on a second 'trip' through the dictionary. I am most likely wrong but that's how I read the code. If this is the case, is there any way to get some 'output' from the sendmail task as to if it's having issues somewhere? Would I get that from debug mode? My theory is taking my 'live data' and getting it onto a dev server of mine, load up the debug compiled code, then testing things out with the sendmail task (such as bad emails, high number of messages, and so on) to try to figure out what's keeping successful messages from being removed if they're sent...
Posted by: tha_watcha - Monday, 16 April 2012 14:03:46
[quote]Thinking back on this, by doing so, I set the guest account to be notified in this manner also, but the guest account doesn't have an email address specified by default, which of course makes sense, but if Yaf is trying to send an email to a blank email address might this not be the cause? [/quote]
No this shouldn't be a problem there is a check if the user is a guest or not before the messages are collected
[quote]The exceptions and errors responses I see in the source code show usage of the "DEBUG" flag. If I compile in debug mode and load it up, where will I find the 'debug' error messages? Do they go to the event log of YAF or are they on screen or what?[/quote]
Yes but currently the code seems wrong normally it should also log email exceptions when not in debug mode but thats not the case. Thats the reason why the event log will be always empty when the send mail threading isnt working.
To finally solve the Problem with the Multiple Mails or Mails not send at all, we need to turn on the logger for this, to hopefully shed some light in to this Problem.
Posted by: bbobb - Monday, 16 April 2012 14:22:19
Maybe a good idea would be to add a host setting which allows debugging output in all modes.
Posted by: francoz - Monday, 16 April 2012 14:31:02
Another 2 cent to this issue: I was facing again multiple email problems even using my new mail server, where initially these seemed to have disappeared.
I have setup the mail server to receive non delivery reports and found that a very active user had a non-existant mail address. After changing this with an updated working one all is fine.
Posted by: tha_watcha - Monday, 16 April 2012 14:58:30
[quote=bbobb;54226]Maybe a good idea would be to add a host setting which allows debugging output in all modes. [/quote]
Agreed, would be easily to add. I add it later.
I enabled the Logger for the Threaded Send Mail Function
Posted by: squirrel - Monday, 16 April 2012 15:41:10
[quote=francoz;54228]Another 2 cent to this issue: I was facing again multiple email problems even using my new mail server, where initially these seemed to have disappeared.
I have setup the mail server to receive non delivery reports and found that a very active user had a non-existant mail address. After changing this with an updated working one all is fine.[/quote]
So are you saying you fixed a bad email address also that was clogging the queue and now it's behaving normally?
Posted by: francoz - Monday, 16 April 2012 17:40:42
[quote=squirrel;54231][quote=francoz;54228]Another 2 cent to this issue: I was facing again multiple email problems even using my new mail server, where initially these seemed to have disappeared.
I have setup the mail server to receive non delivery reports and found that a very active user had a non-existant mail address. After changing this with an updated working one all is fine.[/quote]
So are you saying you fixed a bad email address also that was clogging the queue and now it's behaving normally?[/quote]
Exactly.
My forum has 830 users and it is quite easy to keep under control bad email addresses.
But thinking better about it this can hardly be the issue. The mail server should take the message from the client (YAF Forum application) and LATER try to send it to the recipient, so the queue should not clog for this reason.
I am sure there are no connection issues with my previous mail server (it was the one from my provider, on my same subnet, fiber connectivity, no problems at all connecting from another site via Exchange on a quite busy environment) but there could be some about the way YAF connects to it, for example for too frequent sessions opened.
Just my guess, since there is actually no log to examine.
Posted by: squirrel - Monday, 16 April 2012 17:56:25
[quote=francoz;54232][quote=squirrel;54231][quote=francoz;54228]Another 2 cent to this issue: I was facing again multiple email problems even using my new mail server, where initially these seemed to have disappeared.
I have setup the mail server to receive non delivery reports and found that a very active user had a non-existant mail address. After changing this with an updated working one all is fine.[/quote]
So are you saying you fixed a bad email address also that was clogging the queue and now it's behaving normally?[/quote]
Exactly.
My forum has 830 users and it is quite easy to keep under control bad email addresses.
But thinking better about it this can hardly be the issue. The mail server should take the message from the client (YAF Forum application) and LATER try to send it to the recipient, so the queue should not clog for this reason.
I am sure there are no connection issues with my previous mail server (it was the one from my provider, on my same subnet, fiber connectivity, no problems at all connecting from another site via Exchange on a quite busy environment) but there could be some about the way YAF connects to it, for example for too frequent sessions opened.
Just my guess, since there is actually no log to examine.
[/quote]
See I don't know how the mail task in YAF accounts for communications with the mail server over 'network' - and how it records 'communication' errors. The issue seems to be occurring during communication with SMTP via "network" if there is some form of error transmitting a particular message, the whole queue is left intact in the table instead of the 'successfully' communicated messages being cleared as success is received.
Posted by: JP - Friday, 27 April 2012 20:21:10
Unfortunately the issue still exists with 1.9.6.1 rtw and even beyond...
"It used to work but it is now broken" - How many times have I not said that during my 27 years in the computer business... [smile]
I myself use a Merak mail server on the same machine as the forum as the SMTP relay, and have had this issue after 1.9.4, so I don't really believe in the "bad email address" theory. I get all bounces from bad addresses and do not see any connection between a bounce on eg a watched topic and a sudden barrage of 10 new registrations mails for a single user with the same timestamp...
Could it be that the mail process spawns a new thread for every new sendmail in the queue, and if there is some delay in sending there are several threads sending the same mail, due to small delays in the SMTP contact? Ie reading the same first in queue item from the database? Resulting in parallel work instead of strictly sequential?
Just a crazy idea, since I don't know how it works but love speculating in wild theories based on earlier experiences ... [smile]
Posted by: francoz - Sunday, 29 April 2012 13:27:18
Have to quote JP: removing bad addresses from users' accounts does not solve the problem.
This behaviour is inconsistent: sometimes mailing works fine, sometimes two emails arrive, sometimes five, or six, or seven...
I have changed four mail servers with no apparent effects.
Hopefully developers should investigate on this issue, it is quite annoying for users.
Posted by: pre67vw - Thursday, 3 May 2012 13:51:43
[quote=francoz;54394]
Hopefully developers should investigate on this issue, it is quite annoying for users.[/quote]
Yes! I think it needs to be done as a priority, it's so annoying now that I'm looking for alternatives to YAF.
Posted by: JP - Wednesday, 16 May 2012 21:58:48
My "workaround" is to restart the application via the admin interface, or to clear the app pool in IIS...
Seems to work for the first couple of messages sent by YAF, then the number will rise steadily until it gets a bit random depending on time of day, color of the sky, number of beers in my fridge or whatever...
Had 150 "new registration" mails today, but most were 5-10 copies while some were just 1-3...
I also think this issue should be of priority to our dear developer guys, since also new registrants will be puzzled and think the forum owner is an idiot by sending so many confirmation mails...
I don't mind being called an idiot, but would prefer being taken somewhat seriously as a forum owner... [wink]
Posted by: francoz - Wednesday, 16 May 2012 22:54:06
Here too e-mails are coming in a random number. Even from http://forum.yetanotherforum.net! The issue is really annoying...
Posted by: bbobb - Thursday, 17 May 2012 10:13:54
You'd better ask Jaben as the feature was written by him.
Really it's wriiten in a very intransparent manner.
Posted by: JP - Monday, 21 May 2012 21:59:47
I have received a mail log from my server owner.
Email & host adress names are renamed, but I capitalized name parts so the flow can be followed.
This mail transaction resulted in 3 identical emails.
[code]
193.12.247.11 [16F0] 19:17:56 Connected, local IP=193.12.247.11:25
193.12.247.11 [16F0] 19:17:56 >>> 220-mail.MAILHOST.com ESMTP MAILHOST.com mailserver v10; Mon, 21 May 2012 19:17:56 +0200
193.12.247.11 [16F0] 19:17:56 <<< EHLO MAILHOST01
193.12.247.11 [16F0] 19:17:56 >>> 250-mail.MAILHOST.com Hello MAILHOST01 [193.12.247.11], pleased to meet you.
193.12.247.11 [16F0] 19:17:56 <<< MAIL FROM:
193.12.247.11 [16F0] 19:17:56 >>> 250 2.1.0 ... Sender ok
193.12.247.11 [16F0] 19:17:56 <<< RCPT TO:
193.12.247.11 [16F0] 19:17:56 >>> 250 2.1.5 ... RECIPIENT ok; will forward
193.12.247.11 [16F0] 19:17:56 <<< DATA
193.12.247.11 [16F0] 19:17:56 >>> 354 Enter mail, end with "." on a line by itself
193.12.247.11 [16F0] 19:17:56 <<< 861 bytes (overall data transfer speed=99652778 B/s)
193.12.247.11 [16F0] 19:17:56 Start of mail processing
193.12.247.11 [16F0] 19:17:56 *** 1 856 00:00:00 OK FWM36056
193.12.247.11 [16F0] 19:17:56 >>> 250 2.6.0 856 bytes received in 00:00:00; Message id FWM36056 accepted for delivery
193.12.247.11 [16F0] 19:17:56 <<< MAIL FROM:
193.12.247.11 [16F0] 19:17:56 >>> 250 2.1.0 ... Sender ok
193.12.247.11 [16F0] 19:17:56 <<< RCPT TO:
193.12.247.11 [16F0] 19:17:56 >>> 250 2.1.5 ... RECIPIENT ok; will forward
193.12.247.11 [16F0] 19:17:56 <<< DATA
193.12.247.11 [16F0] 19:17:56 >>> 354 Enter mail, end with "." on a line by itself SYSTEM [2750] 19:17:56 Client session Message id FWM36056 item 201205211917569813.tm$
193.12.247.11 [16F0] 19:17:56 <<< 861 bytes (overall data transfer speed=99652778 B/s)
193.12.247.11 [16F0] 19:17:56 Start of mail processing SYSTEM [2750] 19:17:56 Client session DNS query 'aol.com' 0 (4) [OK - 1] SYSTEM [2750] 19:17:56 Client session Connecting to 'mailin-01.mx.aol.com'
193.12.247.11 [16F0] 19:17:56 *** 1 856 00:00:00 OK FWM37656
193.12.247.11 [16F0] 19:17:56 >>> 250 2.6.0 856 bytes received in 00:00:00; Message id FWM37656 accepted for delivery
193.12.247.11 [16F0] 19:17:56 <<< MAIL FROM:
193.12.247.11 [16F0] 19:17:56 >>> 250 2.1.0 ... Sender ok
193.12.247.11 [16F0] 19:17:56 <<< RCPT TO:
193.12.247.11 [16F0] 19:17:56 >>> 250 2.1.5 ... RECIPIENT ok; will forward
193.12.247.11 [16F0] 19:17:56 <<< DATA
193.12.247.11 [16F0] 19:17:56 >>> 354 Enter mail, end with "." on a line by itself
193.12.247.11 [16F0] 19:17:56 <<< 861 bytes (overall data transfer speed=99652778 B/s)
193.12.247.11 [16F0] 19:17:56 Start of mail processing
193.12.247.11 [16F0] 19:17:56 *** 1 856 00:00:00 OK FWM37756
193.12.247.11 [16F0] 19:17:56 >>> 250 2.6.0 856 bytes received in 00:00:00; Message id FWM37756 accepted for delivery
193.12.247.11 [16F0] 19:17:56 <<< MAIL FROM:
193.12.247.11 [16F0] 19:17:56 >>> 250 2.1.0 ... Sender ok
193.12.247.11 [16F0] 19:17:56 <<< RCPT TO:
193.12.247.11 [16F0] 19:17:56 >>> 250 2.1.5 ... RECIPIENT ok; will forward
193.12.247.11 [16F0] 19:17:56 <<< DATA
193.12.247.11 [16F0] 19:17:56 >>> 354 Enter mail, end with "." on a line by itself
193.12.247.11 [16F0] 19:17:56 <<< 591 bytes (overall data transfer speed=68402778 B/s)
193.12.247.11 [16F0] 19:17:56 Start of mail processing
193.12.247.11 [16F0] 19:17:56 *** 1 586 00:00:00 OK FWM43856
193.12.247.11 [16F0] 19:17:56 >>> 250 2.6.0 586 bytes received in 00:00:00; Message id FWM43856 accepted for delivery
193.12.247.11 [16F0] 19:17:56 <<< MAIL FROM:
193.12.247.11 [16F0] 19:17:56 >>> 250 2.1.0 ... Sender ok
193.12.247.11 [16F0] 19:17:56 <<< RCPT TO:
193.12.247.11 [16F0] 19:17:56 >>> 250 2.1.5 ... RECIPIENT ok; will forward
193.12.247.11 [16F0] 19:17:56 <<< DATA
193.12.247.11 [16F0] 19:17:56 >>> 354 Enter mail, end with "." on a line by itself
193.12.247.11 [16F0] 19:17:56 <<< 591 bytes (overall data transfer speed=68402778 B/s)
193.12.247.11 [16F0] 19:17:56 Start of mail processing
193.12.247.11 [16F0] 19:17:56 *** 1 586 00:00:00 OK FWM45456
193.12.247.11 [16F0] 19:17:56 >>> 250 2.6.0 586 bytes received in 00:00:00; Message id FWM45456 accepted for delivery
193.12.247.11 [16F0] 19:17:56 <<< MAIL FROM:
193.12.247.11 [16F0] 19:17:56 >>> 250 2.1.0 ... Sender ok
193.12.247.11 [16F0] 19:17:56 <<< RCPT TO:
193.12.247.11 [16F0] 19:17:56 >>> 250 2.1.5 ... RECIPIENT ok; will forward
193.12.247.11 [16F0] 19:17:56 <<< DATA
193.12.247.11 [16F0] 19:17:56 >>> 354 Enter mail, end with "." on a line by itself
193.12.247.11 [16F0] 19:17:56 <<< 591 bytes (overall data transfer speed=68402778 B/s)
193.12.247.11 [16F0] 19:17:56 Start of mail processing
193.12.247.11 [16F0] 19:17:56 *** 1 586 00:00:00 OK FWM45556
193.12.247.11 [16F0] 19:17:56 >>> 250 2.6.0 586 bytes received in 00:00:00; Message id FWM45556 accepted for delivery
205.188.59.194 [2750] 19:17:56 Client session Connected, local IP= 193.12.247.11:
193.12.247.11 [16F0] 19:17:56 <<< QUIT
193.12.247.11 [16F0] 19:17:56 >>> 221 2.0.0 mail.MAILHOST.com closing connection
193.12.247.11 [16F0] 19:17:56 Disconnected
[/code]
Posted by: daveburke - Monday, 21 May 2012 23:00:29
I happened to be working in YafSendMailThreaded.cs this afternoon. I was pushing it pretty hard and experienced my first duplicate emails. I was studying bad email address handling by the YAF SendMail task. Very interesting, like bbobb said.
One thing I observed was when multiple emails occurred the mail records seemed to be entering yaf_mail with 1 SendTries attempt (rather than 0.) That shouldn't be possible if you look at the yaf_mail_create sql proc, but perhaps somehow simultaneous calls are made to LegacyDb.mail_create()? Multiple threads? I can't say. Another observation when the duplicates occurred was messages were leaving the yaf_mail table one at a time over several minutes rather than many at a time. I restarted YAF. Processing returned to normal and the dupes were gone.
Sorry I only have observations to report. One thing I did do which might be helpful was modify the yaf_mail_delete SQL proc so that the users of any mail items with two SendTries are placed in a "do-not-mail" lookup table to eliminate possible bad email address processing issues. As for the "do-not-mail" logic, I'm actually sending records to yaf_mail from outside of YAF, which is where I'm checking the do-not-mail status and not sending those to Yaf_mail. It was determined on this thread that bad addresses weren't the culprit in the duplicate emails, but after looking into bad email handling, if nothing else it will reduce processing. And the less a background task has to do is always better.
Thanks,
Dave
Posted by: squirrel - Monday, 21 May 2012 23:18:19
[quote=daveburke;54676]I happened to be working in YafSendMailThreaded.cs this afternoon. I was pushing it pretty hard and experienced my first duplicate emails. I was studying bad email address handling by the YAF SendMail task. Very interesting, like bbobb said.
One thing I observed was when multiple emails occurred the mail records seemed to be entering yaf_mail with 1 SendTries attempt (rather than 0.) That shouldn't be possible if you look at the yaf_mail_create sql proc, but perhaps somehow simultaneous calls are made to LegacyDb.mail_create()? Multiple threads? I can't say. Another observation when the duplicates occurred was messages were leaving the yaf_mail table one at a time over several minutes rather than many at a time. I restarted YAF. Processing returned to normal and the dupes were gone.
Sorry I only have observations to report. One thing I did do which might be helpful was modify the yaf_mail_delete SQL proc so that the users of any mail items with two SendTries are placed in a "do-not-mail" lookup table to eliminate possible bad email address processing issues. As for the "do-not-mail" logic, I'm actually sending records to yaf_mail from outside of YAF, which is where I'm checking the do-not-mail status and not sending those to Yaf_mail. It was determined on this thread that bad addresses weren't the culprit in the duplicate emails, but after looking into bad email handling, if nothing else it will reduce processing. And the less a background task has to do is always better.
Thanks,
Dave
[/quote]
My theory with the bad emails was when YAF got into a process of dealing with many emails -- long running task, that it wasn't completing and those that were stuck were getting resent and resent - like you noted -- you placed the task under heavy usage before you saw the fluke. Since we moved our site to a new server, we get no duplicate emails of any sort. Running on a Win2k8 box, 8 core, 4gb ram hosting around 200 sites, sql2008 r2, etc. Our public mailserver is MailEnable 5, but YAF sends it's outgoing mail to the local IIS SMTP server and it plays middleman to the public mailserver. We used to use our public mailserver directly and did have plenty of issues even though it is hosted on same machine - had many duplicate email issues. Since putting IIS SMTP in the middle and moving to win2k8 the issue has disappeared for us.
Now if I que up a message to get sent to all the users on the site and there are bad emails in there, I guarantee I'll get stuck in a 'duplicate email' issue again until I go in there and clean out the bad email addresses -- then it'll go away.
Keep in mind, this is all on an older codebase than is current. I only offer this information as comparison. We haven't updated to current version just yet -- I've had ton of other projects on the table.
Posted by: Thantis - Friday, 25 May 2012 22:49:41
Well, I am not sure that load issues are the root of the problem, or atleast not the only problem. The forum I am experiencing this on has 13 users :)
Posted by: bbobb - Saturday, 26 May 2012 13:18:47
[quote=Thantis;54711]Well, I am not sure that load issues are the root of the problem, or atleast not the only problem. The forum I am experiencing this on has 13 users :)[/quote]
It's not a load issue. It has problems with logic.At first glance it launches 2 threads instead of 1.
Morover there can be more threads launched if something in sending fails. And the whole thing is not thread-safe, and there are possible double enumerations. Due to the complexity it's impossible to say what exactly happens.
Posted by: JP - Saturday, 26 May 2012 22:44:35
Clearly not a load issue, but rather some kind of issue of sending mails in parallel instead of in a strict sequential order in the mailout-queue... Mail is not time-critical, and should be thread safe in a single thread just reading the mail queue and send the mail once and mark it as sent. Should not have any non-RFC workarounds...
Just wonder why it did not happen in 1.9.4 but started with 1.9.5.x and beyond.... Was there any clever code changes in the mailing module between those versions?
Posted by: Jaben - Tuesday, 29 May 2012 08:03:09
If you're having trouble with duplicate emails, please replace the mail_list SPROC with the following SQL:
[code=sql]
EDIT: doesn't fix the issue.
[/code]
Let me know if that fixes the issues.
Posted by: Jaben - Tuesday, 29 May 2012 15:04:54
Never mind about the SQL recently posted. I have a better solution forthcoming.
Posted by: JP - Tuesday, 29 May 2012 21:38:41
Thanks for looking into it Jaben, rather annoying problem for many of us with 1.9.5 or higher... [thumbup]
All I can say is that myself & members on my site have been quite annoyed by this for a while... I constantly get complaints from members receiving multiple emails, while I myself have grown accustomed to it and see it as a "cosmetic" annoyance.... [wink]
Posted by: Jaben - Wednesday, 30 May 2012 05:52:21
Please run these SQL changes (they can be run in the Admin->Run SQL Query area).
If you have[b] v1.9.6 or greater[/b], you need to run the following SQL:
[code=sql]
ALTER PROCEDURE [{databaseOwner}].[{objectQualifier}mail_list]
(
@ProcessID int,
@UTCTIMESTAMP datetime
)
AS
BEGIN
BEGIN TRANSACTION TRANSUPDATEMAIL
UPDATE [{databaseOwner}].[{objectQualifier}Mail]
SET
ProcessID = NULL
WHERE
ProcessID IS NOT NULL AND SendAttempt > @UTCTIMESTAMP
UPDATE [{databaseOwner}].[{objectQualifier}Mail]
SET
SendTries = SendTries + 1,
SendAttempt = DATEADD(n,5,@UTCTIMESTAMP),
ProcessID = @ProcessID
WHERE
MailID IN (SELECT TOP 10 MailID FROM [{databaseOwner}].[{objectQualifier}Mail] WHERE SendAttempt < @UTCTIMESTAMP OR SendAttempt IS NULL ORDER BY SendAttempt, Created)
COMMIT TRANSACTION TRANSUPDATEMAIL
-- now select all mail reserved for this process...
SELECT TOP 10 * FROM [{databaseOwner}].[{objectQualifier}Mail] WHERE ProcessID = @ProcessID ORDER BY SendAttempt, Created desc
END
[/code]
[b]For v1.9.5.x[/b], please use the following SQL:
[code=sql]
ALTER PROCEDURE [{databaseOwner}].[{objectQualifier}mail_list]
(
@ProcessID int
)
AS
BEGIN
DECLARE @UTCTIMESTAMP datetime
SET @UTCTIMESTAMP = GETUTCDATE()
BEGIN TRANSACTION TRANSUPDATEMAIL
UPDATE [{databaseOwner}].[{objectQualifier}Mail]
SET
ProcessID = NULL
WHERE
ProcessID IS NOT NULL AND SendAttempt > @UTCTIMESTAMP
UPDATE [{databaseOwner}].[{objectQualifier}Mail]
SET
SendTries = SendTries + 1,
SendAttempt = DATEADD(n,5,@UTCTIMESTAMP),
ProcessID = @ProcessID
WHERE
MailID IN (SELECT TOP 10 MailID FROM [{databaseOwner}].[{objectQualifier}Mail] WHERE SendAttempt < @UTCTIMESTAMP OR SendAttempt IS NULL ORDER BY SendAttempt, Created)
COMMIT TRANSACTION TRANSUPDATEMAIL
-- now select all mail reserved for this process...
SELECT TOP 10 * FROM [{databaseOwner}].[{objectQualifier}Mail] WHERE ProcessID = @ProcessID ORDER BY SendAttempt, Created desc
END
[/code]
Please note that v1.9.6 has YAF.NET code improvements that do help deal with these issues.
I've put this code into production here on this forum and have committed it to the latest SQL in codeplex.
Let me know how it goes with your forum.s
Posted by: squirrel - Wednesday, 30 May 2012 06:23:18
I've got an older 1.9.5 forum still running -- I used to have the issue but don't anymore since we migrated servers. We're moving to 1.9.6.1 soon though and it'll have that in there so will be able to report back any issues --
Posted by: daveburke - Wednesday, 30 May 2012 13:43:55
Jaben,
You answered the question I was going to ask, that is, if you applied the updates to the YAF Forums. Because your post was the first time that I can remember when I didn't get multiple notifications! :)
Thanks much,
Dave
Posted by: francoz - Wednesday, 30 May 2012 13:54:55
Working for me, too. [thumbup] I am on 1.9.6.1 version.
Posted by: JP - Friday, 1 June 2012 19:55:11
No duplicates/triplicates/quadruplicates or more the last 2 days after the fix implementation...
Excellent!
Thank you for listening and fixing it. [thumbup]
Posted by: squirrel - Friday, 1 June 2012 22:05:53
I got a duplicate notification for JP's reply.
[code]Received: from mail.tinygecko.com ([38.102.33.228]) by dedicate-it.net with MailEnable ESMTP; Fri, 1 Jun 2012 12:55:51 -0500
Received: from TINY-MANOA (TINY-MANOA [127.0.0.1]) by mail.tinygecko.com with SMTP;
Fri, 1 Jun 2012 13:55:30 -0400
MIME-Version: 1.0
From: "Yet Another Forum.NET"
To: "squirrel"
Date: 1 Jun 2012 13:55:30 -0400
Subject: Topic Subscription New Post Notification (From Yet Another Forum.NET)
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: base64
Received-SPF: none (dedicate-it.net: yetanotherforum.net does not designate permitted sender hosts)
X-ME-Bayesian: 0.000000
Return-Path: [/code]
[code]
Received: from mail.tinygecko.com ([38.102.33.228]) by dedicate-it.net with MailEnable ESMTP; Fri, 1 Jun 2012 12:55:51 -0500
Received: from TINY-MANOA (TINY-MANOA [127.0.0.1]) by mail.tinygecko.com with SMTP;
Fri, 1 Jun 2012 13:55:30 -0400
MIME-Version: 1.0
From: "Yet Another Forum.NET"
To: "squirrel"
Date: 1 Jun 2012 13:55:30 -0400
Subject: Topic Subscription New Post Notification (From Yet Another Forum.NET)
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: base64
Received-SPF: none (dedicate-it.net: yetanotherforum.net does not designate permitted sender hosts)
X-ME-Bayesian: 0.000000
Return-Path: [/code]
Posted by: daveburke - Friday, 1 June 2012 22:23:36
Interesting. I received two copies as well, and two of yours, Squirrel. DOH!
Posted by: Jaben - Saturday, 2 June 2012 02:36:24
This version of the forum has a confirmed problem with the code: it's not really thread-safe. Only v1.9.6 and above has corrected thread-safe code that should only spawn one process.
One day I'll get around to upgrading this forum...
Posted by: squirrel - Saturday, 2 June 2012 05:42:52
No worries Jaben! Just wanted to report back so you had an update on what was happening :)
Posted by: JP - Monday, 4 June 2012 20:22:13
Hmmm... But now I don't get any notifications for PM's & new registrations at all... [confused]
New users are registering, so it seems they get confirmation mails - I hope...
Posted by: bbobb - Monday, 4 June 2012 21:35:04
Does the _mail table contain any rows with a large number of send attempts?
Posted by: JP - Thursday, 7 June 2012 20:48:48
Sorry, my fault.... Registration mails got hidden away in Outlook, too many rules defined.. [blush]
Posted by: Thantis - Thursday, 7 June 2012 22:20:15
This works perfectly for me. Thanks! Now I just need to clear up this bug when creating users and all will be well.
Posted by: nhoward34 - Wednesday, 27 June 2012 20:41:03
The SQL fix has been truncated. I looked at this the other day and it had the full script available.
Can you please repost? I think I am finally to a place where this fix will work - given all of the code updates.
Posted by: squirrel - Wednesday, 27 June 2012 20:49:50
[quote=nhoward34;55127]The SQL fix has been truncated. I looked at this the other day and it had the full script available.
Can you please repost? I think I am finally to a place where this fix will work - given all of the code updates.[/quote]
I've noticed as well the code bbtag sometimes truncates things in this version. Noticed it when sending an PM --
Posted by: nhoward34 - Wednesday, 27 June 2012 20:56:39
Can you re-post the entire SQL script?
Posted by: squirrel - Wednesday, 27 June 2012 21:40:06
[quote=nhoward34;55129]Can you re-post the entire SQL script?[/quote]
Unfortunately I don't have it. I have not applied it on my sites yet, because none of them are having the mail issue since we moved to a new server --
Posted by: nhoward34 - Wednesday, 27 June 2012 22:20:06
Are you able to Force Digest Send from the Admin?
I have the latest code with some recent fixes from tha_watcha (which got rid of the errors), but still is not sending Diest on demand emails.
Just wondering...
Posted by: nhoward34 - Wednesday, 27 June 2012 23:01:01
View the page source and you can retrieve the SQL in its entirety. There is a bad
tag that is causing the issue.
Posted by: squirrel - Thursday, 28 June 2012 04:23:33
[quote=nhoward34;55132]Are you able to Force Digest Send from the Admin?
I have the latest code with some recent fixes from tha_watcha (which got rid of the errors), but still is not sending Diest on demand emails.
Just wondering...[/quote]
We don't use that feature and have it disabled, but I could probably get it loaded on a dev server and do some testing if it was needed --
Posted by: nhoward34 - Thursday, 28 June 2012 16:30:10
I was able to get it all working in v1.9.6.1. Hurray!!!
Thanks for responding to all my posts. I do appreciate it.
Nick
Posted by: tha_watcha - Friday, 29 June 2012 18:45:35
[quote=squirrel;55128][quote=nhoward34;55127]The SQL fix has been truncated. I looked at this the other day and it had the full script available.
Can you please repost? I think I am finally to a place where this fix will work - given all of the code updates.[/quote]
I've noticed as well the code bbtag sometimes truncates things in this version. Noticed it when sending an PM --[/quote]
Just fixed it in the repository, the highlight function inside syntax highlight code was not correctly working
test...
[code=plain;4]
Line 1
Line 2
Line 3
Highlight this line
Line 5[/code]
Posted by: pre67vw - Tuesday, 29 October 2013 16:22:53
It's back.....
I've just done an upgrade to v1.9.6.1 RTW and now the users are getting 8 or 9 notification emails.
Do I have to use the stored procedure fix posted earlier? I thought that RTW would be ready to use?
Posted by: pre67vw - Friday, 1 November 2013 16:55:42
Anyone?
Posted by: bbobb - Friday, 1 November 2013 19:33:42
This is a knowin issue in 1.9.6.1 RTW.
Posted by: Zero2Cool - Friday, 1 November 2013 20:52:30
[quote=pre67vw;61555]It's back.....
I've just done an upgrade to v1.9.6.1 RTW and now the users are getting 8 or 9 notification emails.
Do I have to use the stored procedure fix posted earlier? I thought that RTW would be ready to use?[/quote]
Check out v2.0. It has quite a few good fixes and additions. You will like it!
http://forum.yetanotherforum.net/yaf_postst16247_v2-0-0-RTW-Dated--10-28-2013.aspx