Notes
Atom FeedPID Controller
A PID controller is a common systems control function based on a feedback loop. PID stands for:
- Proportional: A multiplier on the error between the measured and set/target value. This effect increases the farther the output is from the target (error).
- Integral: A multiplier on the accumulated error over time. This effect increases the more time (and error) the output is from the target.
- Derivative: A multiplier on the rate of change of error. This is meant to dampen oscillations.
Below is an example chart that shows a PID controller in a system where the target is “10”, the starting value is “0”, and with a variety of PID tuple configurations.
PermalinkAvailability Percentages
Availability percentages measured in 9s of uptime, qualified by (rounded) expected downtime
| Availability % | Downtime per year | Downtime per month | Downtime per week | Downtime per day |
|---|---|---|---|---|
| 90% | 37 days | 4400 minutes (73 hours) | 1000 minutes (17 hours) | 140 minutes |
| 99% | 3.7 days | 440 minutes (7.3 hours) | 100 minutes (1.7 hours) | 14 minutes |
| 99.5% | 1.8 days | 220 minutes (3.7 hours) | 50 minutes | 7.2 minutes |
| 99.9% | 8.7 hours | 43.8 minutes | 10 minutes | 1.4 minutes |
| 99.99% | 53 minutes | 4.4 minutes | 1.0 minutes | 8.6 seconds |
Source: Wikipedia; High availability
PermalinkJavascript/Typescript Decorators Suck
I just spent a few days looking into implementing Javascript/Typescript decorators into a library and have come to the conclusion that the languages’ implementations are too poorly supported to be distributed in shared libraries.
There seems to be multiple incompatible decorator implementations including:
- An ES2016 decorator of the type
function decorator(target, key, descriptor) - A Typescript 4 decorator
of the type
function decorator(target, key, descriptor)which seems to be mostly the same as in ES2016 but still requires a"experimentalDecorators": trueto be set intsconfig.json - And a Typescript 5 decorator
of the type
function decorator(originalMethod: Function, context: DecoratorContext): Function
All three decorator patterns work by declaring @decorator in front of the class, method, or property to be decoratored.
All three decorator patterns do not work on bare functions.
The first two have also been deemed “experimental” for years (ES2016 has been around for a decade) and are backwards
incompatible with the currently supported decorator pattern in Typescript (not backported to ECMAScript) and therefore
are also poorly supported by compilers like Babel and
various munging tools like webpack.
It’s therefore pretty much impossible for a shared library to use decorators without
resorting to shipping multiple decorator implementations (e.g. a @decorateES2016 as well as @decorateTypescript5) or
having janky instructions like adding flags to node packages’ tsconfig.json.
I guess Javascript is still catching up to Python which has had decorators for 20+ years now.
PermalinkUpgrading MariaDB Database Versions
This script updates the database version for MariaDB docker containers. Given the container name, it will stop the container and launch a new MariaDB container with the updated version of MariaDB but with the original data and network.
Notes:
- The script requires the name of the docker container as an argument
- The new docker container has some typical recommended configurations set like
--cap-add=sys_niceto optimize cpu scheduling. - The script will prompt for the root password in order to execute upgrade scripts
#!/bin/bash
set -exuo pipefail
IFS=$'\n\t'
BASEDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
cd "$BASEDIR/.." || exit 1
NAME="$1"
VERSION="11.4.2"
VOLUME="$(docker container inspect "$NAME" | jq -r .[0].Mounts[0].Name)"
NETWORK="$(docker container inspect "$NAME" | jq -r '.[0].NetworkSettings.Networks | keys[0]')"
echo "$VOLUME"
docker pull "mariadb:$VERSION"
docker stop "$NAME" || true
docker rm "$NAME" || true
docker run \
--detach \
--restart=always \
--network="$NETWORK" \
--name="$NAME" \
--volume "$VOLUME:/var/lib/mysql" \
--cap-add=sys_nice \
"mariadb:$VERSION" \
--character-set-server=utf8mb4 \
--collation-server=utf8mb4_unicode_ci \
--binlog_expire_logs_seconds=3600
docker exec -it "$NAME" mariadb-upgrade -u root -p --force
Concurrent Python Example
Reference example of using concurrent.futures in Python >3.2. Standard
parallel programming warnings still apply (race conditions, sharing state,
inter-process communication):
import concurrent.futures
import time
def calculate(limit: int) -> int:
""" An example function that takes a long time to run """
x = 1
for i in range(1, limit):
x = (x + i) % 10
return x
def iterate(data: list[int]) -> list[int]:
""" A base case without concurrency """
return [calculate(i) for i in data]
def concurrent_submit(data: list[int]) -> list[int]:
""" Concurrency using executor.submit """
result: list[int] = []
with concurrent.futures.ProcessPoolExecutor() as executor:
futures: list[concurrent.futures.Future[int]] = []
for i in data:
future = executor.submit(calculate, i)
futures.append(future)
for future in futures:
result.append(future.result())
return result
def concurrent_map(data: list[int]) -> list[int]:
""" Concurrency using executor.map """
with concurrent.futures.ProcessPoolExecutor() as executor:
# Use list to check for exceptions
data = list(executor.map(calculate, data))
return data
def main() -> None:
""" Execute each of the implementations and time it """
data = [10**7 + x for x in range(20)]
print(data)
"""
# Non-parallel implementation
start = time.time()
output = iterate(data)
duration = time.time() - start
print('iterate: %f' % duration)
print(output)
"""
# Parallel with executor.submit()
start = time.time()
output = concurrent_submit(data)
duration = time.time() - start
print('concurrent_submit: %f' % duration)
print(output)
# Parallel with executor.map()
start = time.time()
output = concurrent_map(data)
duration = time.time() - start
print('concurrent_map: %f' % duration)
print(output)
if __name__ == '__main__':
main()