Albertyw.com

Availability Percentages

February 8, 2025

Availability percentages measured in 9s of uptime, qualified by (rounded) expected downtime

Availability %	Downtime per year	Downtime per month	Downtime per week	Downtime per day
90%	37 days	4400 minutes (73 hours)	1000 minutes (17 hours)	140 minutes
99%	3.7 days	440 minutes (7.3 hours)	100 minutes (1.7 hours)	14 minutes
99.5%	1.8 days	220 minutes (3.7 hours)	50 minutes	7.2 minutes
99.9%	8.7 hours	43.8 minutes	10 minutes	1.4 minutes
99.99%	53 minutes	4.4 minutes	1.0 minutes	8.6 seconds

Source: Wikipedia; High availability

Permalink

Javascript/Typescript Decorators Suck

January 3, 2025

I just spent a few days looking into implementing Javascript/Typescript decorators into a library and have come to the conclusion that the languages’ implementations are too poorly supported to be distributed in shared libraries.

There seems to be multiple incompatible decorator implementations including:

An ES2016 decorator of the type function decorator(target, key, descriptor)
A Typescript 4 decorator of the type function decorator(target, key, descriptor) which seems to be mostly the same as in ES2016 but still requires a "experimentalDecorators": true to be set in tsconfig.json
And a Typescript 5 decorator of the type function decorator(originalMethod: Function, context: DecoratorContext): Function

All three decorator patterns work by declaring @decorator in front of the class, method, or property to be decoratored. All three decorator patterns do not work on bare functions. The first two have also been deemed “experimental” for years (ES2016 has been around for a decade) and are backwards incompatible with the currently supported decorator pattern in Typescript (not backported to ECMAScript) and therefore are also poorly supported by compilers like Babel and various munging tools like webpack.

It’s therefore pretty much impossible for a shared library to use decorators without resorting to shipping multiple decorator implementations (e.g. a @decorateES2016 as well as @decorateTypescript5) or having janky instructions like adding flags to node packages’ tsconfig.json.

I guess Javascript is still catching up to Python which has had decorators for 20+ years now.

Permalink

Upgrading MariaDB Database Versions

June 2, 2024

This script updates the database version for MariaDB docker containers. Given the container name, it will stop the container and launch a new MariaDB container with the updated version of MariaDB but with the original data and network.

Notes:

The script requires the name of the docker container as an argument
The new docker container has some typical recommended configurations set like --cap-add=sys_nice to optimize cpu scheduling.
The script will prompt for the root password in order to execute upgrade scripts

#!/bin/bash

set -exuo pipefail
IFS=$'\n\t'

BASEDIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
cd "$BASEDIR/.." || exit 1

NAME="$1"
VERSION="11.4.2"
VOLUME="$(docker container inspect "$NAME" | jq -r .[0].Mounts[0].Name)"
NETWORK="$(docker container inspect "$NAME" | jq -r '.[0].NetworkSettings.Networks | keys[0]')"

echo "$VOLUME"

docker pull "mariadb:$VERSION"
docker stop "$NAME" || true
docker rm "$NAME" || true
docker run \
    --detach \
    --restart=always \
    --network="$NETWORK" \
    --name="$NAME" \
    --volume "$VOLUME:/var/lib/mysql" \
    --cap-add=sys_nice \
    "mariadb:$VERSION" \
    --character-set-server=utf8mb4 \
    --collation-server=utf8mb4_unicode_ci \
    --binlog_expire_logs_seconds=3600
docker exec -it "$NAME" mariadb-upgrade -u root -p --force

Permalink

Concurrent Python Example

January 1, 2024

Reference example of using concurrent.futures in Python >3.2. Standard parallel programming warnings still apply (race conditions, sharing state, inter-process communication):

import concurrent.futures
import time


def calculate(limit: int) -> int:
    """ An example function that takes a long time to run """
    x = 1
    for i in range(1, limit):
        x = (x + i) % 10
    return x


def iterate(data: list[int]) -> list[int]:
    """ A base case without concurrency """
    return [calculate(i) for i in data]


def concurrent_submit(data: list[int]) -> list[int]:
    """ Concurrency using executor.submit """
    result: list[int] = []
    with concurrent.futures.ProcessPoolExecutor() as executor:
        futures: list[concurrent.futures.Future[int]] = []
        for i in data:
            future = executor.submit(calculate, i)
            futures.append(future)
        for future in futures:
            result.append(future.result())
    return result


def concurrent_map(data: list[int]) -> list[int]:
    """ Concurrency using executor.map """
    with concurrent.futures.ProcessPoolExecutor() as executor:
        # Use list to check for exceptions
        data = list(executor.map(calculate, data))
    return data


def main() -> None:
    """ Execute each of the implementations and time it """
    data = [10**7 + x for x in range(20)]
    print(data)

    """
    # Non-parallel implementation
    start = time.time()
    output = iterate(data)
    duration = time.time() - start
    print('iterate: %f' % duration)
    print(output)
    """

    # Parallel with executor.submit()
    start = time.time()
    output = concurrent_submit(data)
    duration = time.time() - start
    print('concurrent_submit: %f' % duration)
    print(output)

    # Parallel with executor.map()
    start = time.time()
    output = concurrent_map(data)
    duration = time.time() - start
    print('concurrent_map: %f' % duration)
    print(output)


if __name__ == '__main__':
    main()

Permalink

Updating UUIDField on MariaDB to Django 5

December 27, 2023

Updating from Django 4 to Django 5 comes with an incompatible change to how Django UUIDFields are stored in MariaDB databases - in Django 4, Django would store UUIDFields as char(32) types but in Django 5, Django would store UUIDFields as uuid types. However, the Django 5 upgrade notes are incomplete. Following the directions on a nontrivial Django project, it’s still likely that you’ll get database errors like django.db.utils.OperationalError: (4078, "Cannot cast 'int' as 'uuid' in assignment of test_database.table_column.id") and errors from fitting 36-character UUID strings into 32-character database fields.

In order to fix this problem, I found that it’s better to convert char(32) MariaDB fields into uuid fields first before migrating to Django 5. That way, there are fewer changes during the update. In order to do so,

While still on Django 4, replace django.models.UUIDField with a uuid field using a uuid database type with:

# app/models.py

from django.db import models


class RealUUIDField(models.UUIDField):
    def db_type(self, connection):
        return "uuid"


class Model(models.Model):
    id = RealUUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    # Formerly:
    # id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)

Explicitly declare older migrations on the uuid field as a char(36) field. Note that this needs to be char(36) because Django will attempt to store UUIDs with dashes ('06cb5e67-467f-4675-91f3-ca466bcee805' instead of '06cb5e67467f467591f3ca466bcee805'). In migration files, replace models.UUIDField with:

# app/migrations/0001_migration.py
class Char36UUIDField(models.UUIDField):
    def db_type(self, connection):
        return "char(36)"

class Migration(migrations.Migration):
    operations = [
        migrations.CreateModel(
            name='Model',
            fields=[
                ('id', Char36UUIDField(default=uuid.uuid4, editable=False, primary_key=True, serialize=False)),
                # Formerly:
                # ('id', Char36UUIDField(default=uuid.uuid4, editable=False, primary_key=True, serialize=False)),
            ],
        ),
    ]

Run ./manage.py makemigrations to generate a migration that will convert the database column from a char(32) to a uuid type.
Install the django==5.0 pip package. Fix any other incompatibilities.
Replace RealUUIDField back to models.UUIDField.

Permalink