Published in Towards Data Science·Feb 26Member-onlyUse simulations to optimize customer wait time, systems load, and costLet’s say your infrastructure is a fleet of M identical machines, each processing a single task at a time. Each new customer is placed in a queue waiting for a machine to become idle. You know how long each processing task takes to complete, but it’s not a fixed number…Data Science13 min readData Science13 min read
Published in Towards Data Science·Aug 29, 2021Member-onlyData has many periodic components you need to visualize? Treat it like audio!Visualize periodic components using the Fourier transform. — You’re working with timeseries data, and you’re at the initial discovery stage. You’re trying to figure out all the different ways that the data varies in time. There may be trends, whereby the data is constantly increasing or decreasing in time. There may be completely random variations. …Fourier Transform7 min readFourier Transform7 min read
Published in Towards Data Science·Jul 15, 2021Member-onlyBackpropagation from Scratch: How Neural Networks Really WorkHow do neural networks really work? I will show you a complete example, written from scratch in Python, with all the math you need to completely understand the process. I will explain everything in plain English as well. You could just follow along, read just the text and still get…Neural Networks16 min readNeural Networks16 min read
Published in Towards Data Science·Jan 14, 2021Bootstrap statistics — how to work around limitations of simple statistical testsYou have a data sample. From it, you want to calculate a confidence interval for the population mean value. What’s the first thing you think about? It’s usually a t-test. But the t-test has several requirements, one of which is that the sampling distribution of the mean is nearly normal…Data Science7 min readData Science7 min read
Published in Towards Data Science·Dec 28, 2020Member-onlyPfizer and Moderna — vaccine efficacy calculated from dataAs of right now (December 2020), the leading mRNA vaccines for COVID-19 are made by Pfizer-BioNTech and Moderna. Their efficacy is estimated to be most likely around 94% … 95%. — But how was that number calculated? Turns out, the basic value is pretty easy. I’ll show you how to do that, and then I will make an estimate for how confident we are that the value is right. Randomized trials To test a vaccine, you need to do a randomized blind trial…Statistics6 min readStatistics6 min read
Oct 31, 2020Test results, part 2: how about negative results?This is part 2 of this article: Test results are positive. What are the odds you have the virus? Surprise, intuition is wrong! Bayes’s Theorem from statistics applied to pandemicsflorin-andrei.medium.com To recap: there’s a pandemic going on, 1% of people have the virus. There’s a test that can detect the virus, and the test is 99% reliable (for both positive and negative results). But this time, when you take the test, the result…Statistics2 min readStatistics2 min read
Oct 30, 2020Test results are positive. What are the odds you have the virus? Surprise, intuition is wrong!Let’s say there’s a virus pandemic sweeping through the population, and 1% of people have the virus. Let’s say there’s a test for this condition, and the test is 99% reliable, meaning — out of 100 tested cases, the test will be correct in 99 cases, and will be wrong…Statistics5 min readStatistics5 min read
Published in Towards Data Science·Jun 21, 2020Removing non-linear trends from timeseries dataSometimes trends need to be removed from timeseries data, in preparation for the next steps, or part of the data cleaning process. If you can identify a trend, then simply subtract it from the data, and the result is detrended data. If the trend is linear, you can find it…Regression5 min readRegression5 min read
Published in Towards Data Science·Jun 7, 2020The potato train — using Python with extremely large numbers and arbitrary precision for binomial probabilityMath is hard, let’s go shopping — for tutorials, that is. I definitely wish I had read this tutorial before trying some things in Python that involve extremely large numbers (binomial probability for large values of n) and my code started to crash. But wait, I hear you saying, Python…Python9 min readPython9 min read
