Not just you buddy -- I have two more site replicating the behavior. I'm doing some more testing before I can document the full forensics for you. There is a smoking gun here - I just haven't found it yet. This is what I have so far:
1. The two sites I maintain that are exhibiting it are both upgrade sites. One originally a Snitz DB brought into v1.9.3, then brought up to 1.9.5.5, and now 1.9.6.1+revisions. The other one went from 1.9.1.7 to 1.9.3 then 1.9.6.1 then 2.0 beta+revisions. Others that are clean installs after 1.9.5.5 don't exhibit this behavior (so far).
2. Same two sites are housed on different Win2008 R2 servers with SQL Server 2008 R2 as well. Both also run Plesk and MailEnable. One site communicates via 'network' to the MainEnable server - the other connects to the MS-SMTP service (alternate port but still via 'network').
Now onto the wierdness I haven't isolated yet:
One of the sites only randomly seems to send multiple notifications. The other one has more of a pattern.
Pattern on PM notifications (scenario):
Send one PM to another user, they get one notification. If they reply, I got just one. Now, If I reply back, they get 3 notifications. Reply comes back to me, and I get 4 notifications. The more we reply, the more notifications we get. In direct proportion to the number of replies.
On the Post Notifications it's a little different. You get a notification for every reply you have in a thread. So if I participate 4 times in a 30-post thread, I get 4 notifications for every post.
Your site sometimes exhibits similar behavior UNLESS I'm subscribed to a forum that the thread is in. Then I get the proper number of notifications. But if I'm just watching a topic in a forum, I get slammed depending on how many replies are in that topic (follows suit with the PM notification routine I explained above - 1 notification for every reply in a topic (regardless of how many I actually posted. But wait, there's more -- after this topic reached so many replies, it returned to sending the proper number of notifications (just one) (or was it another topic - I cannot remember now - I'll have to check my gmail).
I don't know if this helps you any or makes your tracking it worse. If there's anything else I can to to help, let me know. I don't know if there's a trigger I could attach to my sql database for you and trap the activity (say whenever a notification is added to the mail queue) or what. Just let me know --
If you can't find it using the forum search, try my signature link -- searches this site using Google:
Google is my Friend