Python⇒Speed

Floating point math is often slower than integer math because the compiler is being conservative about how it optimizes your code. While some programming languages already had solutions of a sort, until now Rust did not have a good stable way to deal with

How do you speed up computational Python code? A common, and useful, starting point is: Pick a good algorithm. Use a compiled language to write a Python extension. Maybe add parallelism so you can use multiple CPU cores. But what if you need more speed? C

Imagine you are processing a stream of events, of unknown length. It could end in 3 seconds, it could run for 3 months; you simply don't know. As a result, storing the whole stream in memory or even on disk is not acceptable, but you still need to extract

In a previous post I talked about unit testing for speed, and in particular testing for big-O scalability . The next step is catching cases where you've changed not the scalability, but the direct efficiency of your code. If your first thought is "how thi

When you're building a Docker image for your Python application, you're building on top of an existing image-and there are many possible choices for the resulting container. There are OS images like Ubuntu, and there are the many different variants of the

If your NumPy code is too slow, what next? One option is taking advantage of the multiple cores on your CPU: using a thread pool to do work in parallel. Another option is to tune your code so it's less wasteful. Or, since these are two different sources o

When you implement an algorithm, you also implement tests to make sure the outputs are correct. This can help you: Ensure your code is correct. Catch problems if and when you change it in the future. If you're trying to make sure your software is fast, or

In a recent article by David Lattimore , he demonstrates a number of Rust performance tricks, including one that involve writing code that looks like a loop, but which in practice is optimized down to a fixed number of instructions. Having what looks like

Note: The original version of this article was slightly different, e.g. with 500x speedup; I reworked it to make the argument clearer. If your Python code is slow and needs to be fast , there are many different approaches you can take, from parallelism to

You have a large JSON file, and you want to load the data into Pydantic. Unfortunately, this uses a lot of memory, to the point where large JSON files are very difficult to read. What to do? Assuming you're stuck with JSON, in this article we'll cover: Th