This is where the throughput of an RPC system starts to go off the rails. Then the garbage collector says, “Uh-oh, I guess I better do something about this Gen2 memory.” 🔗Stop the world This memory accrues until the system is under enough load that it can’t allocate additional memory anymore. You essentially have a minor memory leak when you’re invoking remote calls if those calls take enough time to come back. So even if you get a response back from the server and your method completes, your Gen2 memory may not be cleaned up except for IDisposable objects. This is important because the garbage collector doesn’t actively clean up Gen2 memory. The actual timings of the garbage collector’s activity will vary on a lot of things, but the point is that your memory can be put into Gen2 before your RPC call even completes. “Are you done with that memory now?” Your thread is shocked-doesn’t the garbage collector understand how long remote calls take? “No problem,” the collector says, “I’ll come back later.” And it marks your memory as Gen2. It’s also not a long time in terms of a remote call, which is way slower than a local function execution. This is a long time in terms of CPU cycles, but it’s maybe about 50 microseconds for us humans, which isn’t much at all. “No problem,” the garbage collector will say, “I’ll come back and check with you later.” And it marks that memory as Generation 1 (Gen1), so it knows not to bother your thread again too soon.Īround ~50,000 CPU operations later, the garbage collector will come around for a Gen1 memory collection. So in relatively short order, the garbage collector will perform Generation 0 (Gen0) collection, in which it will ask your thread, “Are you done with that memory yet?” Nope, as it turns out, you’re still waiting for a response from the RPC call. Garbage collectors are designed under the assumption that memory should be cleaned up reasonably quickly. Meanwhile, the garbage collector (or whatever manages memory in your runtime environment) is trying to make things efficient by cleaning up memory that’s not used anymore. Otherwise, you won’t be able to use them once you have your response. You may not even be thinking about this as you’re coding, but whatever variables you’ve declared before the RPC call must retain their values. When you begin a remote call, any memory you had allocated needs to be preserved until you get a response back. What happens with threads and memory when you’re doing these remote calls? Even a service that doesn’t turn around and call another service usually has to do something like talk to a database, which is another form of RPC. What you typically see nowadays in a “microservices architecture” using RPC 3 is not a single RPC call, but instead, one service calling another service, which calls another service, and so on. Unfortunately, the web servers serving your RPC request won’t scale linearly forever, which becomes a big problem. Anyone who shows you a graph like that and says “RPC is faster” is either lying or selling you something. There are more steps, so the increased latency is easily explained.īut that’s just a micro-benchmark and doesn’t tell you the whole story. In the messaging case, we need to send a message, write that message to disk, and then another process needs to pick it up and process it. And this makes sense! After all, in the HTTP case, we open a direct socket connection from the client to the server, the server executes its code, and then returns a response on the already-open socket. Initially, the messaging solution takes longer to complete than RPC. Graph of microbenchmark showing RPC is faster than messaging. If you did such a benchmark, here’s an incomplete picture you might end up with: But to be fair, we also have to process the messages on the server before we can consider the messaging case to be complete. It’s tempting to simply write a micro-benchmark test where we issue 1000 requests to a server over HTTP and then repeat the same test with asynchronous messages. It’s less of an apples-to-oranges comparison and more like apples-to-orange-sherbet. Some will claim that any type of RPC communication ends up being faster (meaning it has lower latency) than any equivalent invocation using asynchronous messaging. However, no matter the specific word used, the meaning is the same: remote method calls over HTTP. In place of RPC, 1 they may substitute a different term or technology like REST, microservices, gRPC, WCF, Java RMI, etc. Ignoring all the other advantages messaging has, they’ll ask us the following question: Sometimes developers only care about speed.
0 kommentar(er)
