Blog Post:

If you *must* use .NET System.IO.Ports.SerialPort

Microsoft .NET FrameworkAs an embedded developer who writes desktop software mostly for configuration of, and data download from, peripheral devices, I use serial data streams a lot.  Mostly USB virtual serial posts from FTDI, but also the USB Communication Device Class and real 16550-compatible UARTs on the PCI bus.  Since looking at data through an in-circuit emulator debug interface is generally    a miserable experience, getting serial data communication with a custom PC application is essential to analyzing data quality and providing feedback on hardware designs.  C# and the .NET Framework provide a rapid application development that is ideal for early development that needs to track changing requirements as hardware designs evolve.  Ideal in most respects, I should say.

The System.IO.Ports.SerialPort class which ships with .NET is a glaring exception.  To put it mildly, it was designed by computer scientists operating far outside their area of core competence.  They neither understood the characteristics of serial communication, nor common use cases, and it shows.  Nor could it have been tested in any real world scenario prior to shipping, without finding flaws that litter both the documented interface and the undocumented behavior and make reliable communication using System.IO.Ports.SerialPort (henceforth IOPSP) a real nightmare.  (Plenty of evidence on StackOverflow attests to this, from devices that work in Hyperterminal but not .NET because IOPSP makes setting certain parameters mandatory, although they aren’t applicable to virtual ports, and closes the port on failure.  There’s no way to bypass or ignore failure of these settings during IOPSP initialization.)

What’s even more astonishing is that this level of failure occurred when the underlying kernel32.dll APIs are immensely better (I’ve used the WinAPI before working with .NET, and still do when I want to use a function that .NET doesn’t have a wrapper for, which notably includes device enumeration).  The .NET engineers not only failed to devise a reasonable interface, they chose to disregard the WinAPI design which was very mature, nor did they learn from two decades of kernel team experience with serial ports.

A future series of posts will present the design and implementation of a rational serial port interface built upon, and preserving the style of, the WinAPI serial port functions.  It fits seamlessly into the .NET event dispatch model, and multiple coworkers have expressed that it’s exactly how they want a serial-port class to work.  But I realize that external circumstances sometimes prohibit using a C++/CLI mixed-mode assembly.  The C++/CLI solution is incompatible with:

  • Partial trust (not really a factor, since IOPSP’s Open method also demands UnmanagedCode permission)
  • Single-executable deployment (there may be workarounds involving ILMerge or using netmodules to link the C# code into the C++/CLI assembly)
  • Development policies that prohibit third-party projects
  • .NET Compact Framework (no support for mixed-mode assemblies)

The public license (as yet undetermined) might also present a problem for some users.

Or maybe you are responsible for improving IOPSP code that is already written, and the project decision-maker isn’t ready to switch horses.  (This is not a good decision, the headaches IOPSP will cause in future maintenance far outweigh the effort of switching, and you’ll end up switching in the end to get around the unfixable bugs.)

So, if you fall into one of these categories and using the Base Class Library is mandatory, you don’t have to suffer the worst of the nightmare.  There are some parts of IOPSP that are a lot less broken that the others, but that you’ll never find in MSDN samples.  (Unsurprisingly, these correspond to where the .NET wrapper is thinnest.)  That isn’t to say that all the bugs can be worked around, but if you’re lucky enough to have hardware that doesn’t trigger them, you can get IOPSP to work reliably in limited ways that cover most usage.

I planned to start with some guidance on how to recognize broken IOPSP code that needs to be reworked, and thought of giving you a list of members that should not be used, ever.  But that list would be several pages long, so instead I’ll list just the most egregious ones and also the ones that are safe.

The worst offending System.IO.Ports.SerialPort members, ones that not only should not be used but are signs of a deep code smell and the need to rearchitect all IOPSP usage:

  • The DataReceived event (100% redundant, also completely unreliable)
  • The BytesToRead property (completely unreliable)
  • The Read, ReadExisting, ReadLine methods (handle errors completely wrong, and are synchronous)
  • The PinChanged event (delivered out of order with respect to every interesting thing you might want to know about it)

Members that are safe to use:

  • The mode properties: BaudRate, DataBits, Parity, StopBits, but only before opening the port. And only for standard baud rates.
  • Hardware handshaking control: the Handshake property
  • Port selection: constructors, PortName property, Open method, IsOpen property, GetPortNames method

And the one member that no one uses because MSDN gives no example, but is absolutely essential to your sanity:

  • The BaseStream property

The only serial port read approaches that work correctly are accessed via BaseStream.  Its implementation, the System.IO.Ports.SerialStream class (which has internal visibility; you can only use it via Stream virtual methods) is also home to the few lines of code which I wouldn’t choose to rewrite.

Finally, some code.

Here’s the (wrong) way the examples show to receive data:

Here’s the right approach, which matches the way the underlying Win32 API is intended to be used:

It looks like a little bit more, and more complex code, but it results in far fewer p/invoke calls, and doesn’t suffer from the unreliability of the BytesToRead property.  (Yes, the BytesToRead version can be adjusted to handle partial reads and bytes that arrive between inspecting BytesToRead and calling Read, but those are only the most obvious problems.)

Starting in .NET 4.5, you can instead call ReadAsync on the BaseStream object, which calls BeginRead and EndRead internally.

Calling the Win32 API directly we would be able to streamline this even more, for example by reusing a kernel event handle instead of creating a new one for each block.  We’ll look at that issue and many more in future posts exploring the C++/CLI replacement.

178 Responses

  1. Couldn’t agree with you more about the System.IO.Ports functionality. I tried using it with some success then discovered little quirky problems that you can’t put your finger on, i.e. extreme overrun when using Xon/Xoff handshake. I tried using the built in handshake functions and my own to no avail. Also noted that the reading of the hardware I/O input pins was also unreliable. I am now using the Marshall software libraries that uses the API. Their interface with the Visual Studio environment isn’t great but I have working reliable Comms and will fix the interface when time permits.

    1. Bob, are you using a publicly available software library for wrapping the Win32 serial port API? Or did you mean P/invoke and Microsoft’s System.Runtime.InteropServices.Marshal class?

  2. Dear Ben,

    I eventually stumbled on this post by reading many of your answers on StackOverflow. Clearly you know COM 🙂

    Until now I have often used a particular implementation of IOPSP that is available in the “Termie” project on CodeProject (http://www.codeproject.com/Articles/23656/Termie-A-Simple-RS-Terminal).

    But as it also relies on Read() I have indeed found it to be quite unreliable at times.

    Rolling my own WinAPI wrapper would probably take me too long given my limited experience so I am quite interested to learn more on your take on this approach!

  3. Ben, I am currently in the process of porting a VB6 serial port app to .NET and have found the performance to be inadequate. The app sends a series of messages with responses, before posting the next message. The intermessage gap is several milliseconds larger for the .NET version.

  4. My app is using the DataReceived event, as well as Read(), Write(), and BytesToRead. I am curious if I will see better performance using BaseStream ReadAsync() but not exactly clear on how to handle new data arriving as efficiently as possible.

    1. Reading from the BaseStream will get the data to your application with low overhead. But that’s generally a small part of the processing cost; ultimately how efficiently your serial processing is depends on your code that buffers, packetizes, and parses it.

  5. Could you expand upon line 8 in the recommended approach? Should it work like the IOSPS DataReceived event or something else entirely?

    1. At a high level it is a similar concept to the DataReceived event because it allows the application to respond to incoming data in an event-driven fashion. Important differences are that the event actually carries the received data with it, so the application is all set for processing serial data, and the implementation does this with a single API call, which is far lower overhead.

  6. Could you add more context? I get a compile error on line 13: use of unassigned local variable. I’m using VS 2010 with .Net 4.0.

    BTW, in first comment, IOSPS should be IOPSP.

    1. That looks like a limitation in the compiler’s dataflow analysis; the variable is definitely assigned before use. Initialization by itself isn’t adequate, if the initializer expression passes the variable by reference to another function. But this code doesn’t, it only constructs a delegate using the variable; it doesn’t actually execute it.

      In any case you can work around it by splitting up initialization and delegate creation.

      Action kickoffRead = null;
      kickoffRead = delegate {

  7. Thanks Ben for all the explanation. Do you have any sample code on how to use “ReadAsync” ? I do not seem to find a good document on how to use it.

    Thanks

  8. Can You give full example on sending a message and receiving a response using BaseStream. I have tried this but it is not working for me? Sample would be very helpful

    1. This might be useful:

      for writing, you can use
      await sp.BaseStream.WriteAsync(buffer ,0, buffer.Length);

      for reading,
      byte[] buffer = new byte [7];
      int read = 0;
      while (read < 7)
      read += await sp.BaseStream.ReadAsync(buffer , read, 7 – read);

      these commands should be used within a function that uses "async" as a modifier. For example:
      private async void DoSomething()
      {
      // Read or write like explained earlier
      }

    2. Unfortunately I have to use .NET 4.0. I’m looking for some full example from send to receive an answer. I have sent some data but I have timeout when I’m waiting for response.

      1. If you ask a question on StackOverflow you are welcome to bring it to my attention here. But the blog platform is not designed for Q&A. Besides, at SO you’ll get input from multiple experts. To address your particular case, appropriate timeouts are specific to the device you’re communicating with; I could give sample code that works perfectly in my environment and you could still get timeouts.

    1. Sure, if you set a read timeout of zero and issue a read request, you’ll get as much data transferred to your buffer as is already received (up to the limit of the size you made your buffer). This is a much better approach than querying the buffer size. In general, just getting the data is more efficient and involves fewer race conditions than asking questions about the data and then retrieving it.

  9. As far as I know, readAsync or WriteAsync does not support Readtimeout or Writetimeout. Even creating a manual timeout and setting a condition would not work since await operator is sitting there to get more data, although reading task is complete.

    1. “As far as I know” leads to false assumptions. You can avoid bad assumptions by either testing, or inspecting the implementation logic (JetBrains dotPeek and Red Gate .NET Reflector are good options here). It seems like you did neither.

      There are no serial port functions in the Win32 API that don’t respect timeout. The only way the .NET wrapper would not finish on timeout is if it detected a partial read and looped to read the rest. But I used dotPeek to verify that there is no such logic.

      Stream’s ReadAsync is just a wrapper around BeginRead/EndRead, and that returns once a Win32 ReadFile operation completes. There’s no loop to fill the entire buffer. A timeout will cause the ReadFile operation to complete, which means SerialStream.EndRead is called, and Stream.ReadAsync’s task completes also.

      Now, it is true that SerialStream.ReadTimeout is not as flexible as Win32 SetCommTimeouts. But it does support the desired combination “A (ReadIntervalTimeout) value of MAXDWORD, combined with zero values for both the ReadTotalTimeoutConstant and ReadTotalTimeoutMultiplier members, specifies that the read operation is to return immediately with the bytes that have already been received, even if no bytes have been received.” You get that from setting SerialStream.ReadTimeout=0.

      1. I suspect she tried it or read the documentation for ReadTimeout which says that “This property does not affect the BeginRead method of the stream returned by the BaseStream property.” Unfortunately it seems that BeginRead sets the timeout to infinite before actually kicking off the read and then restores it before returning.

        1. Yeah, Ben Voigt is wrong.

          SerialStream.ReadTimeout has no effect on SerialStream.ReadAsync. I tried setting it to either a positive number or 0 (so it would “return immediately,” as he said). In either case, it just waits indefinitely. As you pointed out, it’s because BeginRead completely ignores the timeout by temporarily setting it to infinity. Here’s the line in the source code for other readers: https://github.com/microsoft/referencesource/blob/master/System/sys/system/IO/ports/SerialStream.cs#L882

          It’s strange how Ben threw in so much irrelevant technical detail. It reminds me of a confidently incorrect AI chatbot.

          1. Perhaps you missed that I was making a statement about how the Win32 API works? It is too bad that Microsoft’s “IOPSP” wrapper over the Win32 API is so horribly broken (which was the motivation of the whole blog post).

            Billy explicitly mentioned, and you alluded to with the word “temporary” that BeginRead puts back the user-specified timeout. What you both glossed over is BeginRead puts back the user-specified timeout while the read is still in-progress. This is a BAD IDEA(tm), and the result is not documented even at the OS level.

            Also, the documentation for ReadTimeout doesn’t mention ReadAsync, even today (and you blame me for not writing in 2014 about what the documentation 6 years later said). Whether or not BeginRead is called from ReadAsync is an implementation detail, not told to the user, and it is up to the implementation to prevent such details from affecting behavior important to the user (such as whether timeouts are ignored). The bugs in this class just never stop.

            Save your accusations of being a chatbot for your own website. Talk about being “confidently incorrect”…

  10. hi Ben, do you know how to properly use SerialPort using .NET compact framework? MSDN said that “The .NET Compact Framework does not support the asynchronous model with base streams.”

  11. Thanks for this posting! Is there anything special you want to note about writing to a serial port? What I’ve gleaned so far is that writing is blocking and should be done on a separate thread.

    1. Writes don’t need a separate thread, because they don’t block as long as there is room in the kernel buffer. If there isn’t room in the kernel buffer, it means you’re writing large amounts of data at a higher rate than the port can drain it — but you configured the port baud rate. (Exception: when the port is only emulating asynchronous serial and baud rate is meaningless) In practice usually writes are interspersed with reading responses, and the write buffer never overflows. Finally, if you’re worried about overfilling the buffer, use WriteAsync rather than using a separate thread.

  12. You originally wrote “A future series of posts will present the design and implementation of a rational serial port interface built upon, and preserving the style of, the WinAPI serial port functions.”

    Are you still going to post those details and implementation ? I’m sure there are plenty of people out there (like me !) who would love to have a more robust alternative to the one Microsoft left us with …

    1. Jason, yes those are still planned, but my dissertation committee asked for some edits before publication so those are a higher priority for my writing bandwidth this month. More serial port blog posts in March. Probably.

  13. great code, I’m just having a problem, when I close the port: serial.close() I recieve an System.InvalidOperationException, “The BaseStream is only available when the port is open” dont know how to close it, I try catch everything and the problem persists, please help me!!! :), thanks

    1. That’s not nearly enough information to understand what’s going on. I suggest you ask a question on StackOverflow, providing all the relevant details including some code snippets and the order of calls. Cancellation of overlapped or async operations may play into your issue as well.

  14. Awesome code, one thing dude, I’m having trouble closing the port using the private void Window_Closing event,

    it throws: System.InvalidOperationException, “The BaseStream is only available when the port is open”. How did you managed to close the stream / port ? thanks

  15. Are the issues described above dependent on the type of serial device (legacy RS232 vs USB)? I know the former can be a bit more complicated (3-wire, 5-wire, 8/9 wire, hardware handshaking, etc.) than the simplified interface that USB provides.

    1. Some of the issues are related to the fact that the software APIs support all the handshaking modes, and many USB virtual serial drivers don’t. (Some do, actually USB is far more complicated than RS-232 even with all the toppings. Virtual COM port drivers attempt to hide that complexity… but often the differences surface to the application, and IOPSP doesn’t handle missing features very well.)

    1. Interesting point. SerialStream (the class implementing BaseStream) has these, but they are inaccessible.

      In the end though, it is a moot point. The SerialPort class appears to make these operations available, but actually makes them useless. (Because events are handled on thread pool threads, there’s a race condition, making the effect of these functions completely unpredictable.)

      The underlying Win32 function PurgeComm is not something you’d ever use while you have background operations in progress. Correct usage is to cancel your background operations, empty the buffers, and start them up again. Which unfortunately you can’t do with the SerialPort class since it always has that WaitCommEvent background operation for event generation, and you can’t do with SerialStream because they aren’t public. So throwing away the wrapper classes and using the Win32 API (possibly through a better wrapper) looks like the only option, if your application really really needs these operations.

  16. I’m pretty much “suffering” with SerialPort as well, especially the fact it deals with the device “suddenly disappearing” while the port is opened…

    A question regarding your code: If I were to use ReadAsync, I would call .ContinueWith(kickoffRead) on the returned Task object to achieve the same as passing kickoffRead as AsyncCallback to BeginRead?

    Also, what do I need to consider when closing the port? Maybe passing a CancellationToken to the ReadAsync method?

  17. Hi Ben,

    Thanks for your profound explanations!

    Our app suffers from sporadic RXOver errors and I suspect the combined use of properties DataReceived and BytesToRead.

    However, I cannot use your approach as we are on Windows CE and Microsoft states in a laconic comment that “The .NET Compact Framework does not support the asynchronous model with base streams.”

    What could be a solution for Windows CE / Compact Framework?

    1. The solution I’ll be presenting is direct use of the Win32 API Communication Functions (which Windows CE does support). Unfortunately, the mixed-mode C++/CLI assemblies I’ll be using to make C# programmers’ lives easier have no equivalent on Compact Framework, so you’d need p/invoke and all the declaration translation concomitant with p/invoking a complex API. The serial port functions themselves aren’t too bad. Detection of port adapter plug and unplug events… well maybe those aren’t of as much concern on .NETCF

    1. My writing calendar, according to priority, is roughly thus: Get dissertation deposited with university, prepare and submit “Supplemental Experience Record” to the Texas Board of Professional Engineers so I can sit for the exam in the fall, continue writing blog entries about serial ports in general, using them under Win32, and the EventDrivenSerial C++/CLI library that I’ve written to supplant System.IO.Ports.SerialPort. So, maybe in a month, and you certainly should see more of my posts this summer.

  18. Hi Ben,

    Would it be possible to somehow get a beta version of your C++/CLI library?

    We could *really* use it on our current project where we are having performance issues (CPU loading mostly) with the standard C# serial port.

    Thanks!

  19. I’ve been researching the use of serial libraries as it pertains to calling from python.

    I’ve had trouble calling dll’s that are written with .net (c# etc). I’m not sure if it’s a managed vs unmanaged thing, but the dlls that are written in C/C++ exports the function names that have no problems being called within python (windows). It looks like I may not be alone:
    https://www.nuget.org/packages/UnmanagedExports

    Any thoughts on this subject? Is there a way to write a windows library (.net or otherwise) that does NOT use System.IO?

    1. Calling from one framework into a serial wrapper library of another framework seems like a bad idea, even if the wrapper weren’t as poorly designed as IOPSP.

      System.IO is not the OS API. It would be very rare to use it from any environment except .NET.

      I suggest either using python’s Win32 API support to call the OS APIs directly or an existing python serial library.

  20. I would like to send some data to the serial device then read the data that it replies with, but for one reason or another I’m not currently able to utilise the async methods or DataReceived event.

    My current implementation is simply a loop that repeatedly calls the Read() method until I’ve received the entire response. It “works”, but I often get a number of TimeoutExceptions (which I just swallow). I assume these are due to the device not replying straight away and/or the reply (which can be many kb) being sent piecemeal.

    Is this an acceptable approach or can you offer any advice for what I’m doing?

    I did wonder if I could avoid the timeouts by waiting until there is actually data at the port (e.g. “while (BytesToRead == 0”) before attempting a Read(), but your article seems to imply that BytesToRead can be problematic?

    1. Busy-waiting is a terrible approach; you shouldn’t do it even if BytesToRead weren’t broken.

      It sounds like your problems arise from the “for some reason or another”. If you’re ignoring timeout exceptions, that means you have set the timeout too short — change it. Use the serial port the way it was designed.

  21. Is it possible to invoke java code from .net based application. If yes, take a look at this new open source library github.com/RishiGupta12/serial-communication-manager seems like it is promising.

    1. This post specifically addresses the case where developers are locked into the framework SerialPort class. When using other libraries is allowed, there are much better options than paying the .NET Java integration cost.

    1. One does not simply upload code without scrubbing all mention of customer-specific requirements and defect reports from the comments. Not even under an “as-is” license.

  22. It’s a good idea to have serial port class in the framework, but I use a 3rd party class from Zyl Soft, which is more stable. There are other 3rd party components as well.

  23. But for configuration issues, which I gather could well be routed through a C++/CLI vernier, which would require, at some level, creating a reader/writer/session using the CreateFile API… Apart from that aspect, everything else could be done through the FileStream, BaseStream, elements? Probably am forgetting bits about it.

    1. You’d think that, but as far as being a wrapper for the underlying API goes, BaseStream isn’t anywhere near complete. Dmitry just ran into another example.

  24. Thank you for the article! It was really great to find it. And I really appreciate your explanation.

    I have several questions about catching Frame error with this approach.

    Context: I have 1Mb one directional stream of data from a device to PC. Chunks are copied for several times, so it is possible to recover if one is lost. Frame error happens from time to time. Frame error usually means that some data is physically lost. But for me it is essential to get as much possible around it.

    ErrorReceived and DataReceived showed complete incompetence, as it stated in your article.

    Using BaseStream approach, at:
    catch (IOException exc) {
    handleAppSerialError(exc);
    }

    Does Frame error show up there?

    Is it synchronous, I mean, do all bytes up to byte when Frame error occurred are received?

    If I continue and call BeginRead after this, would I receive data from the first byte port recovered from Frame error?

    P.S. I can move it to stackoverflow. As it is convenient for you.

    1. Well, ClearCommError is the underlying API for detecting framing errors, and using dotPeek I see that it’s called in three places — the BytesToRead and BytesToWrite properties and the infrastructure for raising the ErrorReceived event on the threadpool. BytesToRead and BytesToWrite clear the status without reading it — obviously a very bad thing if you use them. As far as BaseStream.ReadAsync (or BeginRead) is concerned, you won’t get an exception if you encounter a framing error (which is probably a good thing, since you DID want to get the bytes that were successfully received).

      The really bad thing is that I can’t see anything at all that sets DCB flag bit 14 (fAbortOnError), so you get whatever setting you inherit from the last software that used the port.

      Feel free to ask on StackOverflow, but at this point my diagnosis is that to discover framing errors cleanly, you’ll need to use the Win32 API, either using p/invoke or C++/CLI. If you’re ok with just reading past them, the BaseStream read functions probably do that.

    1. Thanks, I’ve updated the snippet to fix that and the false positive compiler diagnostic concerning an uninitialized local.

  25. Thank you Ben for tip of what to avoid and what to use. Before I was using Thread.Sleep and using the ReadLine/ReadExisting but it felt really unstable and I wanted a general solution which is flexible given some parameters. I have a question regarding when you don’t know the buffer length and the packages might come at different times. Currently I’m doing this (as I know that Carriage Return is the end character of the bytes).

    public async void WriteReadStuff()
    {
    byte[] buffer = new byte[500];
    Port.Write(“w”);
    while (!Encoding.ASCII.GetString(buffer).Contains(ControlCharacter))
    {
    await Port.BaseStream.ReadAsync(buffer, 0, (int) buffer.Length);
    }
    }

    I plan to implement some sort of timeout if the loop runs x number of times then we can say something is wrong. What do you think and do you have any suggestion?

    1. I would not ignore the return value from ReadAsync, which tells you how many bytes were actually placed into the buffer. Also, you probably want to concatenate the incoming partial messages, as well as handle the case when two messages arrived in one read (this might not be possible, if your device uses a request/response protocol exclusively)

  26. Cheers for the quick response. I’m still using Thread.Sleep(1000) between the write and read to get the full answer, otherwise I receive strange results from the electronic device. Thread.Sleep feels like a hack for some reason, since I probably need to experiment with what Sleep time is necessary each time I connect a new device. I also noticed if I send the same message several times (Port.Write(“w”);Port.Write(“w”);Port.Write(“w”)) that it works. Thing is, even when I get wrong result, it still shows that the correct number of bytes have been received.

  27. Thanks Ben for the great article! I’m now using the ReadAsync in my projects.

    What about WriteLine, is it a safe member or I should use WriteAsync?

    1. Writing doesn’t matter as much in the typical case when you don’t care about completion events. A synchronous write will complete when the data is transferred to the kernel buffer anyway, without waiting for the serial transmission. I would still caution you regarding WriteLine, because it accepts a string rather than a byte array, and there’s no standard line termination for all serial devices. Be careful to set your encoding and the NewLine property.

  28. So in your approach you are not using the DataReceived handler at all and just running kickoffRead() in a loop indefinitely?

  29. Ben
    This blog is the closest thing I’ve found for a problem similar to mine…I have a vb.net app (.net 4) which writes serial data (no more than 10 bytes, via System.IO.Ports.SerialPort) and there seems to be a delay in the send. An external device tells me to do something. It takes my program about 15ms to process this request and send the result back to the host. The host is timing me to see how long it takes. I can guarantee that I am sending the serial string within the 15ms, but the host doesn’t get it for 80ms and some times as much as 150ms! What external force is causing this delay? I was original using a USB to serial adapter but have since switched to a PCIe card and am getting the same results.

  30. WOW – I have been struggling with Com time out errors and this seems the issue. I have also been trying to read serial data from a Tinsy serial device that work fine in tera tem but never fires the .Data recieved event handler. I will try you code above.. Thanks Very much…

    1. DataReceived brokenness is race conditions, extra delays, things like that. If you aren’t receiving it at all, then you likely have wrong port settings (check flow control setting especially). Using BaseStream.ReadAsync will avoid the problems inside IOPSP, but you still have to fix misconfiguration.

  31. Serialport works as expected. Use datareceived to populate a concurrent queue along with a background task to read the queue. Works beautifully.

  32. I just spent forever trying to get a serial communication to work. The function here did not help either.
    Turned out that I have to set port.DTREnable=true. I have no idea why I have to do this manually.

    1. Well, you do have to know the serial port settings required by the device you are trying to talk with. Baud rate, parity, stop bits… and in your case the critical piece you were missing is flow control handshaking.

      Until you get those settings right, race conditions in the receive path are a moot point.

      1. Do you know a good guide that covers these things? None of the ones I read mentioned such aspects. I assumed I covered the handshake by setting the port.handshake property.

    2. Lots of sources for information on the different control signals, even wikipedia has it: https://en.wikipedia.org/wiki/RS-232#Data_and_control_signals

      But ultimately you need to know what your particular device is expecting… some require the control signals to be used for handshaking, some require them to be left off, some require faking handshaking (always claim to be “ready”), some require them kept always on so the device can suck a small amount of power. You need device-specific documentation for that, there’s no one-size-fits-all guidance.

  33. Boy, I have hundreds of instruments and robots running on hundreds of apps and couldn’t disagree more. Even USB over RS232. They all work and work robustly. The only thing that is tricky is that the port event runs on a different thread than the main program. But, if you are a decent programmer, you can deal with that.

    1. One thing I forgot to mention is that Framework 2 and 3.5 serial port code runs without modification on Windows XP through 10. Win 8.x and above do not install the C Redistribution Library. So, if you ship a C++ program to customers you need to deal with that, but not if it is .net.

    2. Funny thing about race conditions: you can’t rule them out through any amount of testing. You may be running with DataReceived on a zero load system that’s 1000 times over-spec for the task, and as a result the failure rate due to a race condition may drop down to one every few decades. I feel sorry for your customers who try to use your devices on computers that are doing other things at the same time.

      I have a different definition of “robustness”. In my version, the error events should work (.NET ones are delivered out of order), the software should achieve the full speed of the serial port (making it likely that data continues arriving during your event processing), the software should work on a variety of different computers and even when CPU starved.

      I challenge you to take a look inside the IOPSP code (it’s available online as part of Microsoft’s open source initiatives, as well as easily inspectable using a decompiler) and then see if you still think it’s a reasonable implementation.

      Also, a well-written serial port library shouldn’t need “a decent programmer” to use. It should have a pit of success and just work, getting out of your way so that you can focus your mental energy on the specific features and logic of your application. The .NET implementation fails in this regard, as it requires you to spend time designing thread-safety into your data structures so that events running in threadpool context won’t corrupt things. That’s wasted design time, in every single consuming codebase, that could have been spent on what differentiates your product.

  34. I guess when your software has controlled scientific instruments and robotics and has successfully captured in excess of 35 million data points worldwide by thousands of users with some of the most awful computers while people are looking at YouTube and listening to Internet music with absolutely no issues, that is my definition of robustness. I guess if that you are basing your lamenting comments on your experience, you should probably learn how to create professional programs and not blame the technology. (Editor’s note: removed profanity)

    1. If you control the device firmware, you can work around any number of problems in the desktop code… but not efficiently. If you need to transfer data continuously and long-term at the full baud rate, you can’t afford buffering and go-back-N ARQ. 35 million data points? Please, I have more samples in a single data set. Often in communication with devices over which I have no control, so implementing retries in the application protocol is not possible.

      And you completely avoided addressing the point about time wasted on thread-safe designs, when you could do the entire transfer, with lower CPU utilization, and trivially and provably-correct thread-safety, just by using a single thread (overlapped I/O never blocks the thread).

      And it isn’t just my opinion, I linked to multiple documented cases of IOPSP being completely incapable of exchanging data with some devices at all.

      So, continue to take your chances. With your easy-to-meet requirements, you’ll probably do well enough. Meanwhile I’ll use the API that doesn’t introduce gratuitous reordering of events and throw unrecoverable exceptions in absence of any severe failure.

  35. Trust me, we are an FDA regulated global manufacturer. If the issues you describe were true, all regulatory Hell would break loose.

    I would never consider your solution.

    My .net code is actually a C# port of a high performance event driven, multithreaded, overlapped C++ dll and it works every bit as good.

    It handles sustained data receives at 115.2k baud.

    As I look at your code, I can see why you have issues. There are at least two glaring issues with it. Rather than attacking me, I would suggest that your time would be better spent thinking about your proposed solution and fixing it. That would have value for your users.

    That said, am I supposed to believe that you googled it and that is your support? Are you kidding? 99 percent of web sites are BS.

    I only came to your site because a colleague pointed it out to me.

    I probably should have not visited it.

    This is my last word on this site and I won’t be returning. So, don’t waste your time typing a response. I certainly won’t be.

    Reader: Beware.

    1. I’m familiar with the FDA software guidance. You couldn’t possibly use .NET in an application that the guidance applies to. Perhaps in your test and calibration software.

      If you had tried to use it in a regulated device, you would have had to perform a risk analysis and code review. Just as I found issues by reading the code, you would have as well. But you haven’t, and instead of doing so, you set up a strawman argument involving Google searches. Meanwhile, your own evidence in favor of using IOPSP appears to be entirely based on testing. Again, I will remind you what you already know — you cannot test quality into a multi-threaded application. Testing is not reliable for detecting the presence of race conditions.

      I wasn’t attacking you, I was attacking your choice to use a buggy framework and particularly for trying to use my blog post to spread your incomplete information. However, you’ve now chosen to up the ante by implying that .NET’s built-in serial port code, the System.IO.Ports.SerialPort class and helper code supporting it, is suitable for FDA regulated software. I will give you the benefit of the doubt by extending you a chance to clarify that implication — I truly hope that you were simply trying to throw around supposed expertise from familiarity with regulated code, and that you don’t actually use it in any medical project. Because IOPSP in particular, and .NET in general, are completely unsuited for use in any life support system, or medical data system that presents data to be used for making medical decisions. SOUP (Software of Unknown Provenance) can’t have quality tested into it. You need code analysis.

      So just come out and admit it — the products you have using .NET are neither medical devices nor medical device data systems, and are exempt from regulatory processes. And working for an FDA regulated manufacturer says nothing about the projects you don’t apply the same level of scrutiny to.

  36. Not only my company, but many other FDA regulated companies use it. Abbott, GE Healthcare, and Roche to name three. I learned how to program In 1966 and have been working in a GMP environment since 1973. So, don’t try to preach to me, let alone make assumptions of me spinning rhetoric when you, yourself, admit you have no FDA experience whatsoever. That said, your guess at how things are actually validated, documented, and released are totally wrong. You need to look at 21 CFR before you make those kinds of statements because they really aren’t flattering to your credibility.

    This is getting boring. I’m out.

    1. So you work for one of those groups that takes the attitude “CFR 21 doesn’t actually require us to use the FDA guidance documents, we’ll develop our own process.” Better get into compliance with “Guidance for Industry, FDA Reviewers and Compliance on Off-The-Shelf Software Use in Medical Devices” (and the others) now instead of waiting for the inevitable FDA audit.

      Yes, I do suggest that you quit commenting now, because if you continue without clarifying what class of device you use .NET and IOPSP in, you look like you have something to hide, and if you do clarify, and it’s unregulated software, you reveal yourself as deceitful, and if you clarify that you are using it in a medical device, then the Texas professional engineering rules would leave me no option but to report the situation (rule 137.55).

  37. Nice article – certainly forced me to look at things again!
    I have an application using the DataReceived event on the SerialPort class to buffer data before validating the response and sending the next command (no flow control!!) – basically for automated loading of configurations to embedded TI micros running some custom firmware..

    It’s a total mess of procedural code that I’ve never been happy with, but it worked at the time and I was on a tight delivery deadline, but I’m revisiting now.

    Where’s that up-vote button…

    1. This post is 1 1/2 years old and he still hasn’t posted any more details on a proper serial port implementation… sadly. I don’t think we’ll ever get it.

      1. Ben is probably quite busy. That being said, I would be more than happy to actually ‘crowdfund’ an initiative here… I think a good library would actually be very valuable to the community.

  38. This is a great insight into potential serial port problems with.NET so thank you Ben.

    At the moment I am using the receivedData event and I am experiencing some long delays (> 100ms)

    I know Ben is very busy but I was hoping that someone within the community may have a working example of how to use this approach within a C# .NET 4 environment (or .NET 4.5 would be OK).

    Can anyone help/advise?

    Many thanks

  39. I’ve been using .Net’s SerialPort for years, reading and writing binary data at 115,200 baud; with no problems. Without missing any data from packet sizes up to 65,000. I have used the DataReceived event, and BytesToRead in hundreds of programs with no errors.

    1. Nothing surprising there — many devices are designed to avoid the unusual settings / dark corners of the UART, where the worst of the .NET bugs lurk. As long as you aren’t interested in having framing errors reported correctly, are ok with high overhead and useless use of worker threads, then the sample code can be “good enough”. Especially if you pile application-layer checksums and retry logic on top, then it doesn’t matter if the underlying wrapper returns wrong results for BytesToRead — you can read only part of the buffered data and wait for the next DataReceived event to keep going.

      Of course, that doesn’t work so well if the design requires accurate timestamping on incoming data, fast turnaround, or (and there are some devices in the wild that do this) use intentional framing errors as a signalling mechanism. Those are horrible approaches that should be dealt with via sender timestamps in the data stream, etc. But the UART and Windows kernel drivers do give you the tools you need to deal with them, if you code against the Win32 API directly and not a broken wrapper.

      An important point is that your experience of stuff “working well enough” does nothing to prove absence of bugs. If analysis of the source code reveals bugs (and it does), then testing that says “no bugs” doesn’t prove the analysis wrong, it proves the testing inadequate. Please don’t misunderstand, I’m not discounting your empirical data, but your usage is not designed to find bugs as an exhaustive test would be. So just don’t use it to draw conclusions.

  40. Im expereincing problems with my code … Im getting a System.IO.Ports error when i execute my code on my moxa. Originally, I was using my ReadByte but since reading this article, I attempted to change it but still getting there error. Can you please help me implement it using your way? Here’s my “READ” portion of the code:
    public void SendData(ref int temp2, SerialPort _serialPort)
    {
    try
    {

    string c = Convert.ToString(temp2);
    byte[] array_out = Encoding.ASCII.GetBytes(c);
    _serialPort.Write(array_out, 0, array_out.Length);
    byte[] array_out2 = new byte[1];
    array_out2[0] = 0xD;
    _serialPort.Write(array_out2, 0, array_out2.Length);

    _serialPort.Write(array_out, 0, array_out.Length);
    _serialPort.Write(array_out2, 0, array_out2.Length);

    int reader = 0;
    string xstring = string.Empty;
    Console.WriteLine(“Byte Sent”);
    while (true)
    {
    Console.WriteLine(“Start Read”);
    reader = _serialPort.ReadByte();
    Console.WriteLine(“Bytes Read”);
    char xchar = Convert.ToChar(reader);

    if (xchar == ‘r’)
    {
    if (ProcessLine(xstring, ref temp2) == true)
    {
    if (temp2 == 100)
    {
    _serialPort.Close();

    }
    break;
    }

    xstring = string.Empty;
    }

    if (xchar != ‘r’)
    xstring += xchar;
    }
    }
    catch (Exception ex)
    {
    Console.WriteLine(ex);
    }

  41. Im expereincing problems with my code … Im getting a System.IO.Ports error when i execute my code on my moxa. Originally, I was using my ReadByte but since reading this article, I attempted to change it but still getting there error. Can you please help me implement it using your way? Here’s my “Write & Read” portion of the code:
    public void SendData(ref int temp2, SerialPort _serialPort)
    {
    try
    {

    string c = Convert.ToString(temp2);
    byte[] array_out = Encoding.ASCII.GetBytes(c);
    _serialPort.Write(array_out, 0, array_out.Length);
    byte[] array_out2 = new byte[1];
    array_out2[0] = 0xD;
    _serialPort.Write(array_out2, 0, array_out2.Length);

    _serialPort.Write(array_out, 0, array_out.Length);
    _serialPort.Write(array_out2, 0, array_out2.Length);

    int reader = 0;
    string xstring = string.Empty;
    Console.WriteLine(“Byte Sent”);
    while (true)
    {
    Console.WriteLine(“Start Read”);
    reader = _serialPort.ReadByte();
    Console.WriteLine(“Bytes Read”);
    char xchar = Convert.ToChar(reader);

    if (xchar == ‘r’)
    {
    if (ProcessLine(xstring, ref temp2) == true)
    {
    if (temp2 == 100)
    {
    _serialPort.Close();

    }
    break;
    }

    xstring = string.Empty;
    }

    if (xchar != ‘r’)
    xstring += xchar;
    }
    }
    catch (Exception ex)
    {
    Console.WriteLine(ex);
    }

  42. Is it possible to have a timeout on the SerialPort.BaseStream.ReadAsync function? When it does not receive data it hangs forever, ignoring my set timeout.

    My current code (from https://social.msdn.microsoft.com/Forums/en-US/b22ed8e7-4504-460a-b489-d99c81d51866/how-would-i-go-about-making-an-async-implementation-for-extension-methods-to-the-serialport-class?forum=async):

    var ret = string.Empty;
    // Asynchronously read one byte at a time until our returned string contains the delimiter value.
    while (!ret.Contains(SerialPort.NewLine))
    {
    var bytesRead = await SerialPort.BaseStream.ReadAsync(ByteBuffer, 0, ByteBuffer.Length);
    ret += Encoding.UTF8.GetString(ByteBuffer, 0, bytesRead);
    }
    return ret.Substring(0, ret.Length – SerialPort.NewLine.Length);

  43. With all the ruckus on this post recently, I’m actually really curious – what, specifically, are the underlying issues with DataReceived and BytesToRead?

    The one I understand, as a programmer coming from the embedded angle, is that there is no way for the consumer of new bytes (the event handler) to signal that it has, in fact, consumed the “new” bytes, and no guarantee of the timing of these “new” bytes through DataReceived. BytesToRead is bad because it clears any errors, and is inherently a useless coding pattern (because we don’t know when it’s updated).

    Now, a long time ago, I indeed ran into problems with DataReceived and BytesToRead. I’m curious if the solution I cooked up is actually correct, as I’m not nearly as familiar with how the native serial port is supposed to be used. Basically, I had a parser thread which would wait for a flag to be set by DataReceived, after which it’d attempt to read off bytes and parse them, and then go back to sleep when there were no bytes left. I didn’t want any error handling. What would happen, of course, was that occasionally, at certain data rates, and at certain data burst intervals, and certain parsing strategies (i.e., read a single byte or several at once), DataReceived wouldn’t actually trigger, because of course some race condition happens and the read thread will get stuck waiting for a trigger (that came too early) and BytesToRead looked like there was no new data when the thread went to sleep. My solution was to add a timeout which would wake up the thread for some predicted amount of time, based on expected data rate, in case this happened.

    My question is, is that logic sufficient to bypass any incorrectness in the serial port library?

    It’s possible that this “FDA regulated” software guy has some convoluted logic like this that can fix the symptoms of this race condition. I’ve been rewriting my core libraries to match what you’ve found right now, but I actually suspect it’s possible to make something lower performance but still functional using just Read(), BytesToRead, and DataReceived (please let me know if I’m wrong – I’m really curious). I recall seeing this problem actually at lower data rates and fast bursts of data, and while reading certain number of bytes at once, so being able to run your code continuously at 115.2kbaud really isn’t a great reliability test. Re using the library in high-reliability applications – I can totally believe it “passed” 35 million data points without “error” – but the times the serial port library failed, the user may have passed it off as a “bad connection” and fixed the problem by restarting equipment (which I see people do all the time). Be careful with just relying on testing though – bugs like this are how Toyota’s UA bug killed a bunch of people when those race conditions happen in just the wrong place…

    1. I’m a 61508 certified engineer. To be honest I have no care for framing errors, parity checking etc. It is irrelevant. The fact is that when you develop functionally safe communications you incorporate the require mechanisms to detect failures. Which in simple terms are CRC checking, Redundancy in duplication, and stale data prevention (sequence counters), and cycle rate (data too fast too slow, and so on. You can do this all with byte read. its pretty binary. Of course I wouldn’t do it with .net, but I do know of many certified systems running on java etc. Raised eyebrows and all that aside, it can be done with .net if you are perverse enough. Heck, if the linux kernel can be used for safety systems, then lets just use VB6!

      Ps, I’ve never really had any issues with the .net serialport class. but yeah, its not really full featured as one would like for modern use like events and async etc.

  44. I realize this is an old post, but I just tried out this code and I think there’s a memory leak somewhere. With the code as above, as soon as I connect to the serial port, the memory usage skyrockets, and the application becomes unresponsive and I have to stop debugging it.
    After kneading the code a bit I got it to run smoothly, but there’s still a small memory leak in there somewhere. When running in debug I notice that the number of objects in the Visual Studio Diagnostic Tools keep increasing every time data is received, and it never seems to go down again.

    This is my revised code:

    byte[] buffer = new byte[MAX_RECEIVE_BUFFER * 3];

    Action kickoffRead = null;
    kickoffRead = delegate {
    _serialPort.BaseStream.BeginRead(buffer, 0, buffer.Length, delegate (IAsyncResult ar) {
    try {
    int bytesRead = _serialPort.BaseStream.EndRead(ar);
    byte[] received = new byte[bytesRead];
    Buffer.BlockCopy(buffer, 0, received, 0, bytesRead);
    lock (_receiveBuffer) {
    _receiveBuffer.AddRange(received);
    }
    received = null; // Resetting this has no effect, but left in to rule it out.
    } catch (IOException) { }
    kickoffRead();
    }, null);
    };

    kickoffRead();

    1. Glenn, I think the leak is in the code you added. Perhaps you are calling kickoffRead() multiple times, each of which creates a buffer? Because it has iteration built-in, you should only call it once per serial port. Or perhaps your _receiveBuffer.AddRange(), which was not part of my code, is growing endlessly.

    2. kickoffRead() is only called in the code listed above and nowhere else.

      _receiveBuffer is where I put all the data which I go through to see if a complete message has been received. Whenever a message has been receievd this buffer is emptied (_receiveBuffer.Clear()) so it should clean up after itself.

      Maybe there’s something in the System.IO.SerialPort itself?

  45. hi Ben. Thanks for this article.
    Do you ever get errors coming from BaseStream.EndRead ‘The I/O operation has been aborted because of either a thread exit or an application request’?
    It doesn’t seem to affect the application if i just handle the error and ignore it. However i wonder if it affects performance.

  46. hi Ben. thanks again for your post.
    I used your code and it works quite well. However, I get many errors from BaseStream.EndRead – ‘The I/O operation has been aborted because of either a thread exit or an application request’. These occur frequently between successful reads. I handle these and ignore them but i’m worried there is something that i’m doing wrong. Any comments?

  47. Thanks for this post Ben.
    I have my code working fairly well now. However the basestream.endread throws many exceptions ‘The I/O operation has been aborted because of either a thread exit or an application request’. These errors are thrown frequently in between successful reads. Should i be concerned?

    1. Seems notifications of new comments got queued for a while and I just received a whole batch. Anyway… it’s not normal for I/O operations to just get aborted for any reason except timeout.

      If you enable “native debugging” then the Visual Studio debugger should log thread creation and termination events. Seeing whether these aborts are correlated with worker threads exiting could rule out thread exit as a cause.

      What timeouts have you configured on this port, and does your incoming data stream go silent long enough to cause timeouts?

      Normally .NET has a different exception for timeouts, but remember that we’re using the BaseStream directly and not going through the convenience wrappers. The benefit is that things still work when the assumptions of the wrapper get violated. The downside is that any conversion of error codes into specific exceptions done in the convenience wrapper is also skipped.

      1. Thanks for this great post.

        I sometimes get the IOException (‘The I/O operation has been aborted because of either a thread exit or an application request.’) also when BaseStream.EndRead gets called. After reading through these comments, I traced it down to the DiscardInBuffer I call every time before the SerialPort.Write(). This may be the same reason others are getting the same exception.

  48. I see this thread is still fairly active, so I’m going to ask… I am using the .NET serial port class and the .WRITE method is definitely blocking when writing to the output buffer of my serial port.

    In Jan 13, 2015, you stated “they don’t block as long as there is room in the kernel buffer”.

    Running the following simple test, I see that the .WRITE is blocking:

    logPrint(“Starting write to Serial Port driver.”)
    UUTSerialPort.Write(bTXbuf, 0, bTXbuf.Length) ‘1000 bytes in buffer
    logPrint(“Finished writing to Serial Port driver.”)

    My port is open at 9600 BAUD, Odd parity, 8 data, 1 stop bit and other defaults. And port is using the default .WriteBufferSize = 2048.

    At 9600BAUD, each 11-bit character takes 1.146ms to transmit and from the timestamps in my logPrint function, I see it took 1025 ms to send the 1000 bytes to the serial port driver before the .WRITE returned.

    Since 1000*1.146=1146ms, it really seems that this is writing one byte at a time to the output port.

    The ONLY possible explanation I can think of is this is being caused by the USB-emulated serial port. I am using a B&B USB serial port.

    Any thoughts?

    TIA
    Chip

    1. Chip. 1000 bytes is definitely greater than the hardware buffer size. Most serial ports, including USB virtual serial ports, have a configurable buffer size found in the Advanced tab of the port’s properties (accessed from Device Manager).

      It appears that you’re using the Write method on the S.IO.P.SerialPort class, which is what I recommended NOT to do. So that method may be waiting for the port to drain its buffer, even though the actual write at the Win32 level completed instantly.

      Try using BaseStream’s Write method, and if you still see that the write buffer isn’t being used, there’s BeginWrite / EndWrite, or the newer Task-based WriteAsync. Those should definitely allow your code to make progress while the buffer feeds the serial port, without having to manage multithreading and thread synchronization concerns yourself.

  49. I am using the .NET serial port class and the .WRITE method is definitely blocking when writing to the output buffer of my serial port.

    Running the following simple test, I see that the .WRITE is blocking:
    logPrint(“Starting write to Serial Port driver.”)
    UUTSerialPort.Write(bTXbuf, 0, bTXbuf.Length) ‘1000 bytes in buffer
    logPrint(“Finished writing to Serial Port driver.”)

    My port is open at 9600 BAUD, Odd parity, 8 data, 1 stop bit and other defaults. And port is using the default .WriteBufferSize = 2048.

    At 9600BAUD, each 11-bit character takes 1.146ms to transmit and from the timestamps in my logPrint function, I see it took 1025 ms to send the 1000 bytes to the serial port driver before the .WRITE returned.

    Since 1000*1.146=1146ms, it really seems that this is writing one byte at a time to the output port.

    The ONLY possible explanation I can think of is this is being caused by the USB-emulated serial port. I am using a B&B USB serial port.
    Any thoughts? TIA

  50. I was previously using the example(wrong) to read Maxsonar ultrasonic sensor readings in Mono C#.
    Quite often the UI thread was frozen while getting the values from the serial port.
    It’s been 2 days since I changed it to BaseStream approach. Haven’t seen any frozen UI yet.
    Thank you.

  51. Greeting Ben. Your article really helped me a lot while I was trying to troubleshot the serial communication problems in my real-world application (embedded system, time critical). It proved all the read methods are not really reliable to handle the communication of embedded system.

  52. Hi Ben,
    first thanks for a very interesting read, all the way back to August 2014 indeed. I particularly liked the flame war between you and Lagos Mihali. I’m on your side! I truly deeply understand the angst all those using System.IO.SerialPort. What a complete and unmitigated crock of festering donkey bollocks if there ever was one. However, you pain comes not not even close to the anguish felt by the unwashed compact framework brethren (flails self with a scourge made of old rs-232 cabling and d9 connectors). I have been grappling with this loathsome coprolith for 5 years, and I can tell you, I’m going to need a lot more medication.

    Thanks Again!

  53. I’m doing this for my thesis project and it isn’t particular clear where that code is supposed to go. Is there ever going to be a time where an example of how this works can be posted? An example that is complete, compliable code?

    Thanks

      1. Yes, that would be great honestly. I’ve struggled for months trying to get it to work correctly and I’m not real familiar with async.

  54. Hello, how are you, I tell you that I have a problem that I still can not solve, I have a system that connects to a usb port that converts a 485 to usb, in order to send and receive data, the problem is that at some point the system It stops working and we see that the port is blocked, we tried:
    Clean the port, ask if the port is closed, if the thread that sends the information continues sending or not, but everything is working well and the port does not respond, after this we download our program and use the terminal and neither send nor receive, The last tried is the ReadTimeout, writeTimeout, but I still do not know how it really works.

    Can you give me some advice please be very helpful.
    We use the SerialPort C # component of .NET

  55. Hi every body
    and thanks to Ben Voigt.
    I want to send and receive data through the communication device class usb device in C#. I came across an strange problem: when I send data to the usb device, only one byte has been received by the usb device or maybe I say one byte has been sent from the c# program. Can anybody guide me about this problem? I spend a lot of time to understand what happen, however I didn’t succeed.
    this problem also occurs when I send data using some terminal software for example Herculs terminal software. however, I find one terminal software which could send and receive correct. I think it is written by a low level language like C or Delphi.
    any comments are valuable.
    thanks

  56. ‘@Jacob and others that are interested
    FYI, The comment by Michael above was somewhat helpful to me is getting a better idea of how the code fits in the bigger scheme of things. Although the code may have a problem as he describes, I think you can get an idea of how it is supposed to work. I will be trying it out myself in the next few days so if I am successful will come back and post more info.

  57. I have a funny problem…I have some software that is working with lots of computers (over 100) It connects to a serial port using the .NET serial port class. The software must connect at 115200 for all the features to work. Now normally there is no issue. However, I have two computers that cannot connect at 115200. They can connect at 19200 or 9600 but not 115200. In addition, they can connect at 115200 with a few other pieces of software without issues! This is what is happening when looked at with a serial port sniffer:

    19200 Baud Trace:
    [29/04/2018 19:05:26] – Open port COM3

    [29/04/2018 19:05:26]Changed baudrate: 19200

    [29/04/2018 19:05:26]Disabled RTS

    [29/04/2018 19:05:26]Disabled DTR

    [29/04/2018 19:05:26]Changed line control: Databits 8, Parity none, StopBits 1

    [29/04/2018 19:05:26]Changed flow control: ControlHandShake 0x00, FlowReplace 0x00, XonLimit 55546, XoffLimit 55546

    [29/04/2018 19:05:26]Changed baudrate: 19200

    and this is at 115200

    115200 Baud trace:
    [29/04/2018 19:10:43] – Open port COM3

    [29/04/2018 19:10:43] – Close port COM3

    [29/04/2018 19:10:43] – Open port COM3

    [29/04/2018 19:10:43] – Close port COM3

    Here is the code:

    port = new SerialPort(comport, baud);
    port.ReadTimeout = 20;
    port.Open();

    I use the .NET serial port class in a couple of pieces of software with over 2000 users. It’s been in use for 5 years and this is the first time i have run into an issue.
    Any ideas what could be going on?
    Thanks

  58. Thank you so much for this Ben, the article was very usefull.
    After few adjustments to my application everything was working great!
    I also used this method to write to the COM port, just make sure that you invoke EndWrite otherwise you will have memory leaks (as I did).

  59. Hi sir … i have use for sending sms from C# application this two …methods to connect the serial port..
    clsSMS port=new clsSMS();
    !; this.port = objclsSMS.OpenPort(tlsCmbSMSconnect.Text, Convert.ToInt32(“19200”), Convert.ToInt32(“8”), Convert.ToInt32(“300”), Convert.ToInt32(“300”));
    ……
    public static SerialPort CommonPortName=new SerialPort();
    2: clsSMS.CommonPortName = objclsSMS.OpenPort(tlsCmbSMSconnect.Text, Convert.ToInt32(“19200”), Convert.ToInt32(“8”), Convert.ToInt32(“300”), Convert.ToInt32(“300”));
    the 2nd object i take in all apps as globle objecbt, but not working and create exception. the first work in one form, how i take use the globle serial port the 2nd..

  60. After switching to BaseStream.ReadAsync I found that data read has Unicode encoding despite the setting I have
    _serialPort.Encoding = Encoding.GetEncoding(28591);
    The encoding used to work just fine with _serialPort.ReadExisting()
    The issue is that characters >= 0x80 are not converted correctly back to decimal. E.g. ASCII character 0x80 is converted to 0x20ac ‘€’
    Is there a way to bypass that encoding and stay with extended ASCII?

    1. Data read from a serial port is just a raw byte sequence. There are no characters. There is no interpretation of encoding.

      Whatever encoding problems you are having, are a result of the layer converting between bytes and characters, not the serial port layer.

      For example, Encoding.GetEncoding(28591).GetString(bytes_from_the_serial_port) will apply the encoding of your choice.

  61. Hi Ben, thank you for keeping up with this article over the years. I was hoping you might be able to help me. My problem is vaguely related to this topic. I am developing a program and recently acquired a large number of proprietary devices that connect to a USB receiver (think wireless audience polling). I am trying to use these devices in my program, but have not been able to read from the devices. They are not sending ascii codes, I know that. I also know that they are function properly. They work in the original proprietary software and I can see their output in Device Monitoring Studio. Is there something I might be missing? Is Visual Studio C# able to do this without custom drivers/DLLs? Any help would be greatly appreciated! Side note: the SDK is $800 🙁

  62. Ben,

    Thanks for this. About to implement a app that reads a stream from Serial port and this will avoid a lot of headache. Has anything changed in the .Net implementation of later releases or is this still the best approach IYP?

    1. Steven,

      The only real development since 2014 is that UWP provides a new and completely different/incompatible serial port API (Windows.Devices). It looks like all development effort went there, but I haven’t used it that one enough to know if it’s viable or not.

      The System.IO.Ports namespace in the framework remains the same as ever, and if you’re going to use it I still recommend using completion of an async I/O call as your data arrival signal, and totally avoiding the C# events. The previously “new” development was the addition of ReadAsync, which has the same advantages as BeginRead/EndRead, but is more convenient (as long as you are comfortable with async/await).

      1. It should also be pointed out that the ReadAsync when called with a CancellationToken will completely eat the token unless it’s already cancelled and your read task will hang forever!

        You can verify this yourself by looking at the Stream.ReadAsyc base implementation which simply wraps a call to BeginEndReadAsync without the token.

        The Windows SerialStream implementation does NOT override this default functionality, although the new Linux implementation does.

  63. I program occasionally in VB.NET and barely know C#.

    Would you please explain why kickoffRead is initially declared to be an Action and set to null but then gets set to be a delegate?
    Also, kickoffRead is a delegate which calls itself?

    Would explain this code snippet in more detail?

    Thanks.

  64. When you say DataReceived and BytesToRead are “completely unreliable” — can you detail what you mean by that. I see you say lower that you can call BytesToRead and then when you actually go to read the bytes there are more in the buffer than BytesToRead reported (because they came in between the BytesToRead call and the data retrieve call). But besides that, what are the other gotchas?

    1. Michael, the first and most severe is that DataReceived fires on a threadpool thread, and can be fired again without waiting for the previous event handler to return. So it leads you into a race condition, where when you go to read the buffer there are *fewer* than BytesToRead promised you, because another instance of the event handler read them in the meantime. The application programmer can overcome that by explicitly synchronizing.

      But synchronization in the application won’t solve the race condition that exists within the implementation itself. BytesToRead calls ClearCommError to get the buffer level, and discards everything else. But ClearCommError is an atomic exchange on a number of error flags — you get one shot at seeing them then they’re gone. And other code in the framework is looking at those flags to trigger PinChanged and ErrorReceived events. Because BytesToRead ignores them, events get lost. In fact, the MSDN page for the ErrorReceived event says that “Because the operating system determines whether to raise this event or not, not all parity errors may be reported.” This is an outright lie — the loss of events happens inside the getter functions for BytesToRead and BytesToWrite.

      Its been a few years and my notes on the bugs are not handy, so there are probably a couple more problems with both BytesToRead and DataReceived that I am not remembering today.

      1. Ben, thanks for the reply. I don’t have experience with likelihood that BytesToRead could drop PinChanged, ErrorReceived, or any other flag in an actual application, or that such issues would affect my actual application in particular. But I have gone ahead and followed your advice. Below is a class-ified version of your code, in case it’s helpful to anyone. (I’m assuming the blog will flatten all the indentation. If you think the code is good, perhaps you can indent it.) Please let me know if you see any problems in the code. The idea is the calling program would provide the dataReceivedAction and serialErrorAction, and would do its own locking around whatever data structure it accumulates the bytes into in dataReceivedAction, which can then get locked/retrieved later on its main thread. One important addition to the code is the check for the port being closed before calling EndRead(). From what I could see (and see the linked stackoverflow topic), it was necessary to make this check, or you’d get an exception when calling EndRead after closing the port — and I didn’t see any more reliable way to “cancel” the BeginRead() — even the cancelToken of the newer BeginAsync() method didn’t sound reliable from what I read..

        public class SerialPortReader
        {
        // http://www.sparxeng.com/blog/software/must-use-net-system-io-ports-serialport
        // https://stackoverflow.com/questions/6759938/stop-stream-beginread
        SerialPort _port;
        byte[] _buffer;
        Action _dataReceivedAction;
        Action _serialErrorAction;
        Action _kickoffRead = null;
        IAsyncResult _ar;
        public SerialPortReader( SerialPort port, int bufferSize, Action dataReceivedAction, Action serialErrorAction )
        {
        this._port = port;
        this._buffer = new byte[ bufferSize ];
        this._dataReceivedAction = dataReceivedAction;
        this._serialErrorAction = serialErrorAction;

        this._kickoffRead =
        delegate ()
        {
        this._port.BaseStream.BeginRead(
        this._buffer, 0, this._buffer.Length,
        delegate ( IAsyncResult ar )
        {
        this._ar = ar;
        try
        {
        if ( !this._port.IsOpen )
        {
        // the port has been closed, so exit quietly

        return;
        }

        int actualLength = this._port.BaseStream.EndRead( ar );
        if ( actualLength > 0 )
        {
        byte[] received = new byte[ actualLength ];
        Buffer.BlockCopy( this._buffer, 0, received, 0, actualLength );
        this._dataReceivedAction( received );
        }
        }
        catch ( Exception e )
        {
        this._serialErrorAction( e );
        }

        this._kickoffRead();
        },
        null
        );
        };

        this._kickoffRead();
        }
        }

  65. Hi Ben,
    Thanks for this informative article.
    I am currently experiencing some problems with SerialPort IOException “The requested resource is in use”.
    We have an application which is connected to a modem via serial port. As part of our testing, we have disabled the modem while the application is running. In effect this is like simulating that the modem suddenly does not work and disappears. So when we enable the modem again, our application is not able to connect to the same serial port with the error stated above. This could possibly be a .NET issue but closing and disposing of the serial port does not have any effect. The only solution is to restart the application.
    Hoping you could give me some assistance.
    Thanks in advance.

  66. I’am surprised that you did not mention a huge bug in the SerialStream class!
    The function
    internal void Write(byte[] array, int offset, int count, int timeout)
    takes a timeout but this variable is NEVER used inside the function!

    I have set a SerialPort.WriteTimeout = 2000;
    and observer the write operation hanging FOREVER.

    How can Microsoft produce such a SHIT ?

  67. I found out that the timeout for SerialPort.Write() when the transfer is blocked by an XOFF works correctly on my PCI card with Serial ports.
    But on my USB to RS-232 adapter CH340 (“USB2.0Ser!”) the timeout is completely ignored.

    So it seems that the buggy crappy chinese driver is not working correctly.
    However if Microsoft would have implemented an additional timeout in the SerialPort class in the Write() function this would not happen.

  68. We never were happy with the IO.Ports.SerialPort class and luckily found the ADONTEC SuperCom library for NET. It also includes a “SerialPort” clone class. So we had very little to rewrite and it runs rock solid. If any one interested see at adontec.com.

    1. Cloning the SerialPort class and replacing the implementation can only go so far, for it to actually work you really do have to replace the interface/abstraction as well. So I can’t agree with any recommendation of a replacement that adopts the API of IO.Ports.SerialPort.

  69. The kickoffRead delegate should take a reference to port.BaseStream and then use it in place of port.BaseStream for the BeginRead and EndRead calls. This is because if the port is closed while an asynchronous read is pending, the call to port.BaseStream for EndRead will cause a NullReferenceException since BaseStream is set to null in the Dispose(bool) method of the port. This will also leak your buffer that’s pinned in memory from the BeginRead call, causing your heap to become fragmented over time.

    1. Well spotted Tim, that’s a great way to make sure the EndRead call can be made and cleanup the buffer and associated state (although nothing stops the stream from unpinning outstanding requests when it’s closed, it’s not a good idea to rely on that).

    1. The compiler’s definite assignment analysis doesn’t allow a self-referential delegate to be created in one step (well, not without Y-combinator tricks that would increase code complexity). This issue is related to the fact kickoffRead needs to use itself as a continuation callback — it’s not true recursion.

    2. Because overlapped read and discard buffer both act on the input buffer, it should be no surprise that they conflict. What made you think repeated DiscardInBuffer() calls were good?

    3. I took a quick look and there are quite a few things that I don’t like about that implementation (receives one character at a time, forces caller to handle threading, loses error flags, copies data unnecessarily into unnecessary unmanaged allocations) and additionally is generally an archaic non-C# style. I recommend staying away.

  70. Wow! I’ve tried about 15 other ways to do what you did, and NONE of them worked, from asyncRead(), to .Read(), to whatever other solution there is. I was happy and troubled to find that my data problems were not my fault. The way you have setup this piece is pretty obtuse to me for now, but it is the only way that I get reliable, consistent and complete serial data. THANK YOU SO MUCH for taking the time to share this solution. Once I decided to finally try to understand the basics of your solution, I discovered just how easy it is to pop it into my code. It’s neat that it async’s itself and you can just run it from a set of sync’d code and it will just keep on tickin’.

    Thanks again

    1. Sadly no, it looks like the same bug-ridden class just published in a new location. A couple of spot checks didn’t find any fixes. For example, BytesToRead still discards the error state of the port without processing the corresponding error events.

  71. Sadly I’ve not had much success with this method either.
    For one thing on exiting the application the kickoff delegate causes an exception.

    I have a simple protocol to implement and the timing is not working.
    Send a request to a device (at present implement on an Arduino)
    Wait for the response, the longest of which is a string of 220 character.
    Bisect the string and process the data.
    send the next request.

    Basically it is a synchronous process but for the read to be async makes it easier to look for an abort event.
    Even though only one device is connected there is always garbage in the response (which works correctly with a terminal).
    and that exception on exit.

    It’s been far too long since I implemented serial port comms and I have forgotten most of it by the looks.

    1. In order to gracefully exit (or close the port at any time), you will want to make the recursive call to kickoffRead conditional. You’ll need some sort of flag that you set when stopping the data stream before calling Close on the port object. Then if that flag is set, ignore the exception and exit from the completion callback without calling kickoffRead again.

  72. Hey Ben,
    I like many found my way to this blog post because of the System.IO issues.
    I do mostly Embedded systems programming and hardly any windows based. That was the reason I used
    Visual Studio 2019 because I figured it would be easy to get up and running and was instead disappointed.
    I have a quick question for you.
    I put the code you provided right after Serial.Open which is inside of a button that is clicked to connect. I had these issues
    on compile.

    byte[] buffer = new byte[blockLimit]; <– This tells me blockLimit is undefined
    raiseAppSerialDataEvent(received); <– This throws a error saying it doesn't exist
    handleAppSerialError(exc); <– This has the same issue doesn't exist.

    I'm using Visual Studio 2019 and don't know if that is causing the issues because I tried to figure out why raiseAppSerialdataEvent(received);
    doesn't exist but I couldn't find much information. Also forgive me as I'm not fluent in Visual Studio .net and could be the reason I'm not
    understanding what is going on.
    Thank you in advance

    1. You must provide those, that’s why they have “app” in the name. This is an example of how to stream data from a serial port. No processing of the data is included in the example (it cannot be, as the necessary processing varies wildly depending on the higher-level protocol implemented in the device)

  73. Hi Ben,

    I’m trying another way around to read serial port after it being write and stumble upon your blog. Not knowing where your code should be placed, I’m placing your code after I write to serial port. First flow, ok. Second flow, my window hang and I got BSOD with stop code: DPC_WATCHDOG_VIOLATION. I have few way that I try to read from serial port. From DataReceived event, to Async TaskCompletionSource, BaseStream.ReadAsync, Monitor.Pulse, AutoResetEvent just to read. But your code give me BSOD. Can your validate whether my method is correct if I want to use your code?

    public async Task WriteToPort(byte[] dataToWrite, int Timeout)
    {
    bool result = false;
    try
    {
    if (m_port == null)
    return false;
    OpenPort();
    if (m_port.IsOpen)
    {
    string bufferHex = string.Join(“”, dataToWrite.Select(c => String.Format(“{0:X2}”, Convert.ToInt32(c)).PadRight(4, ‘ ‘)));
    OnPortStatus?.Invoke(“Write bytes to port. Timeout(sec): ” + Timeout);
    m_port.DiscardOutBuffer();
    m_port.DiscardInBuffer();
    if (m_port.Handshake == Handshake.None)
    {
    m_port.DtrEnable = true;
    m_port.RtsEnable = true;
    }

    m_port.Write(dataToWrite, 0, dataToWrite.Length);
    OnDeviceDataSent?.Invoke(bufferHex);
    byte[] buffer = new byte[200];
    Action kickoffRead = null;
    kickoffRead = delegate
    {
    m_port.BaseStream.BeginRead(buffer, 0, buffer.Length, delegate (IAsyncResult ar)
    {
    try
    {
    int actualLength = m_port.BaseStream.EndRead(ar);
    byte[] byteRecv = new byte[actualLength];
    Buffer.BlockCopy(buffer, 0, byteRecv, 0, actualLength);
    OnPortStatus?.Invoke(“STREAM HEX: ” + string.Join(“”, byteRecv.Select(c => String.Format(“{0:X2}”, Convert.ToInt32(c)).PadRight(4, ‘ ‘))));
    OnDeviceDataArrayReceived?.Invoke(byteRecv);
    }
    catch (System.IO.IOException exc)
    {
    WriteLog.ErrorLog(exc.Message, exc.StackTrace, System.Reflection.MethodBase.GetCurrentMethod().Name);
    }
    kickoffRead();
    }, null);
    };
    kickoffRead();
    result = true;
    }
    }
    catch (Exception ex)
    {
    WriteLog.ErrorLog(ex.Message, ex.StackTrace, System.Reflection.MethodBase.GetCurrentMethod().Name);
    }
    finally
    {
    if (m_port != null)
    {
    if (m_port.Handshake == Handshake.None)
    {
    m_port.DtrEnable = false;
    m_port.RtsEnable = false;
    }
    }
    }
    return result;
    }

  74. Dear Ben,

    Have you tried to create a PR with changes to fix the issues in the code posted in .Net Core 5.x repository on GitHub?

    RIchard

  75. Nice post. I study something more difficult on totally different blogs everyday. It can always be stimulating to read content from other writers and practice a bit of something from their store. Thanks for sharing.

  76. Hi Ben,

    Thank you so much. I was really beginning to question my sanity.

    Am modding a Unity game, using Harmony, and need to connect to an ESP32, initially over COMMS. I have WiFi working no problem.

    System.IO.Ports.SerialPort worked fine, in it’s own little project. A soon as I introduced it to the mod it stopped communicating, made no sense at all..

    Am not sure I understand it (delegate) entirely yet, but I have it working and gives me something to learn upon.

    Really, Thank You ! I owe you a beer, actually make that several 🙂

  77. Hi Ben,

    You mentioned in the article: “The Read, ReadExisting, ReadLine methods (handle errors completely wrong, and are synchronous)”

    Could you share a bit more on how are errors being handled wrong?

  78. The example code provided appears to be recursively calling kickoffRead(); i.e. calls delegate from within the delegate. Surely this will eventually cause a stack-overflow error?

    1. It looks that way at first glance, but it’s not “real” recursion. There isn’t an accumulation of stack frames because each time kickOffRead() is called, it returns right away (since all it does is call BeginRead(), which also returns right away). The next call to kickOffRead() is from the callback at some later time.

  79. Ben, this articule is a piece of art! The concepts you explain here are the foundations of the Valentin Gies implementation and work great. Thank you again for this article! Have a nice day!

  80. Hello,

    I realize it is really old post but I partially used your solution recently and noticed there is an issue “Offset and length were out of bounds for the array or count is greater than the number of elements from index to the end of the source collection.”
    in line Buffer.BlockCopy(buffer, 0, received, 0, actualLength);
    port.BaseStream.BeginRead(buffer, 0, buffer.Length, delegate (IAsyncResult ar) {

    Buffer.BlockCopy(buffer, 0, received, 0, actualLength);

    }

    When CPU is busy with other processes or we use a break point in other part of our application which is multithread it leads to this exception.
    On my side i solved it by decreasing the input param “count” to half.
    port.BaseStream.BeginRead(buffer, 0, buffer.Length / 2, delegate (IAsyncResult ar) {

    Buffer.BlockCopy(buffer, 0, received, 0, actualLength);

    }

  81. “The Read, ReadExisting, ReadLine methods (handle errors completely wrong, and are synchronous)”
    if i use read in a synchronous way, could you please explain on handle errors completely wrong? the amount of data i expect to receive is small, can i still use it with caution for accurate reading?

  82. Wow.
    I’m mostly an embedded C programmer, so C# is pretty far out of my comfort zone, but I have written reasonably complex programs with it. However I was stymied when trying to read high speed data over serial-USB (80,000 bytes/sec) using the ‘correct’ method. It would fail with a few seconds of starting to read data. I ran across this blog and basically cut and pasted the code into my program and took my best guess at the data handling routine. Dang if it didn’t work right off!

    Thanks!!!!!

Leave a Reply

Your email address will not be published. Required fields are marked *

Get in Touch

If you have a product design that you would like to discuss, a technical problem in need of a solution, or if you just wish you could add more capabilities to your existing engineering team, please contact us.