Email services and "pipelining"

xanatos

I'm running a Spinneret and emailing data via my isp. Everything was working perfectly, until... the data just stopped after a few weeks.

Turns out the isp updated their anti-spam software and the smtp conversation that the Spinneret performs was flagged as being "pipelining".

Anyone ever heard of this?

Long story short, I set up a new email with a different provider and it was working well... until noon today. I'm guessing.........

Does anyone know if there are any isps that will reliably allow these kind of automated machine-generated emails? It's ok to pay, as long as passage is guaranteed.

Thanks,

Dave

Heater.

xanatos,

email pipelining is described in RFC 1854. http://tools.ietf.org/html/rfc1854

That document has some examples of mail sessions with and without pipelining enabled. You may be able to figure out how to ensure you are not pipelining from that.

stamptrol

I've been doing this for nearly a year from multiple sites using a gmail account for the email "From" address. No problems other than the odd "email failed" when the satellite internet connection gets fuzzy.

The systems are the generic version of the old PINK webservers from Netburner.

Cheers,

xanatos

Hi Tom,

What ISP are you using for the mailserver that you connect to?

Also, what's been suggested to me is that the program code is only waiting for a fixed delay of 200ms between SMTP commands, not actually waiting for a server response, and so my commands are coming too quickly. A test program was created that significantly increased the intercommand delays, and the email went through. Perhaps this is my answer - I'll (hopefully) find out Friday when I can get on their network again.

Here's an example of the SMTP conversation section of the code I'm using now:

    'Send greeting
    StringSend(id, string("EHLO xanatos@xanatos.com", 13, 10))          
    pause(wait)
      
  '==============================================================================
    StringSend(id, string("AUTH LOGIN", 13,10))
    pause(wait)
   
    'Send user name                                                        
    StringSend(id, string("esaeXRvc0Bwger5hdG9zLmNvbQ==", 13, 10))          
                                                                             
    'Send password
    StringSend(id, string("Rea1dAta83m0v3d0U812uqT==", 13, 10))          
  '==============================================================================
   
    ' From Address
    StringSend(id, string("MAIL FROM: xanatos@xanatos.com", 13, 10))       
    pause(wait)

All I should need to do is replace the pause statements with something that waits for a server response, correct?

Thanks,

Dave

Heater.

xantos,

Exactly.

Looking at the RFC I linked to above we see that the normal email conversation is:
1) Send something.
2) Wait for response.
3) If you don't get an error and there is more to send, go to 1)

The idea being you don't proceed to the next step until you know the server has handled the previous step.

It shows that in pipelining you don't wait. Just send a load of stuff and check the responses later.

If you are not so worried about occasional failure then not checking the responses and using a delay (It's there even if you don't have it explicitly in your code) is OK. That is basically what you have been doing until now.

xanatos

Thanks Heater. I also saw in there that I can use HELO instead of EHLO. If I increase my delay to 500ms from 200 it seems to work, but being that I'm picky about things, I'm going to write it so that it actually waits for the server response... just not quite sure yet how that would be accomplished, although the answer to that question is likely already exemplified somewhere in the rest of the code... :-)

Heater.

I think you are right to be picky.

Having code full of odd delays to get it to work is always a clue to me that something is not quite right, often seriously wrong.

You might find a delay length that makes it work, mostly, but then might fail at random times when the network round trip is exceptionally long.
Making the delay longer might make it work more of the time but then you are slowing things down unnecessarily. Perhaps not an issue in this case.
Waiting for responses and having a time out on those waits speeds things up and you will know why it failed if it ever does.

Mike G

The original Spinneret SMTP examples uses a delay. To Heater's point, the code has problems. Mostly due to my inexperience at that time. I tried to take better care of the handshakes in the latest version.

Xanatos, while the attached uses the newer libraries, you should be able to copy and paste to your current project. Please note, the attached code DOES NOT verify a positive response. It only verifies a response, good or bad, and writes the results to the PST. Let me know if you need help parsing the response.