[{"content":"","date":"3 July 2026","externalUrl":null,"permalink":"/","section":"Alex Haslam","summary":"","title":"Alex Haslam","type":"page"},{"content":"","date":"3 July 2026","externalUrl":null,"permalink":"/tags/machine-learning/","section":"Tags","summary":"","title":"Machine Learning","type":"tags"},{"content":"","date":"3 July 2026","externalUrl":null,"permalink":"/tags/mlops/","section":"Tags","summary":"","title":"Mlops","type":"tags"},{"content":"","date":"3 July 2026","externalUrl":null,"permalink":"/tags/open-source/","section":"Tags","summary":"","title":"Open Source","type":"tags"},{"content":"","date":"3 July 2026","externalUrl":null,"permalink":"/tags/python/","section":"Tags","summary":"","title":"Python","type":"tags"},{"content":"At work we build various sklearn models in production to solve time series problems in industrial settings. These models get trained, versioned, and deployed across different environments. The standard way to save a fitted sklearn model is pickle, but this is far from ideal.\nThere is the obvious security concern: loading a pickle executes arbitrary code. It is also not stable across sklearn versions. You can\u0026rsquo;t see what is inside the model as the binary blob is opaque. However, the biggest problem was that we build wrapper classes around the sklearn estimators to add extra functionality we need, and these were also included in the pickle. This meant that any refactor of our code could break previously saved models, so backwards compatibility was a constant pain. This wasn\u0026rsquo;t workable in the long term.\nExisting tools don\u0026rsquo;t cut it # Hugging Face make skops, an existing library for saving sklearn models in a safe binary format. This solves the security issue but the output is still an opaque blob.\nWhat\u0026rsquo;s ironic is that PyTorch doesn\u0026rsquo;t have this problem, even though the models are far more complex. state_dict() gives you a clean dict of tensors, and safetensors handles the rest. Separating weights from structure is part of PyTorch\u0026rsquo;s design, but unfortunately this is not the case for sklearn. The fitted state is scattered across various private attributes with no standard way to extract or restore it.\nRolling our own # We ended up solving this problem internally, by building our own serialization that decomposes a fitted estimator into JSON (hyperparameters) and safetensors (weights). Over time it grew to cover most of the sklearn ecosystem, including lightgbm and xgboost. Those two were the easy part, since they already serialise to JSON natively; most of the effort went into the sklearn estimators, which don\u0026rsquo;t.\nAt some point I realised this was a generic serialisation layer which could be useful to others. The serialization code already had no dependency on our domain code, so it was quick and easy to extract.\nThe result is skeights1:\ncarbon-re/skeights Serialize fitted scikit-learn models to safetensors + JSON. No pickle. Python 3 0 It exposes a simple API to save and load models:\nimport skeights skeights.save(fitted_pipeline, \u0026#34;model.safetensors\u0026#34;, \u0026#34;model.json\u0026#34;) loaded = skeights.load(\u0026#34;model.safetensors\u0026#34;, \u0026#34;model.json\u0026#34;) It works out of the box with the sklearn-like estimators most people use, which is what we\u0026rsquo;ve covered so far.\nA worked example # Say you train a simple pipeline; a Ridge regression with some scaling on top:\nfrom sklearn.linear_model import Ridge from sklearn.pipeline import Pipeline from sklearn.preprocessing import StandardScaler import skeights pipe = Pipeline([ (\u0026#34;scaler\u0026#34;, StandardScaler()), (\u0026#34;model\u0026#34;, Ridge(alpha=0.1)), ]) pipe.fit(X_train, y_train) skeights.save(pipe, \u0026#34;model.safetensors\u0026#34;, \u0026#34;model.json\u0026#34;) The hyperparameters and structure end up in model.json, which you can read at a glance:\n{ \u0026#34;model_params\u0026#34;: { \u0026#34;steps\u0026#34;: { \u0026#34;scaler\u0026#34;: { \u0026#34;copy\u0026#34;: true, \u0026#34;with_mean\u0026#34;: true, \u0026#34;with_std\u0026#34;: true, \u0026#34;type\u0026#34;: \u0026#34;sklearn.preprocessing.StandardScaler\u0026#34; }, \u0026#34;model\u0026#34;: { \u0026#34;alpha\u0026#34;: 0.1, \u0026#34;fit_intercept\u0026#34;: true, \u0026#34;solver\u0026#34;: \u0026#34;auto\u0026#34;, \u0026#34;type\u0026#34;: \u0026#34;sklearn.linear_model.Ridge\u0026#34; } }, \u0026#34;type\u0026#34;: \u0026#34;sklearn.pipeline.Pipeline\u0026#34; } } while the fitted arrays go into model.safetensors:\nscaler/mean_ (3,) float64 scaler/scale_ (3,) float64 model/coef_ (3,) float64 model/intercept_ () float64 Models as config # The major advantage is that a model is now (mostly) JSON, so you can build and manipulate models as data rather than code. A hyperparameter sweep becomes a matter of generating configs instead of instantiating estimators in Python.\nIt also composes nicely. We save the model config as just one field within a larger JSON object alongside the rest of our config, rather than having to handle a separate binary artefact on the side.\nColumnar tree serialization # While lightgbm and xgboost natively serialize to text or JSON, storing the entire model structure this way has drawbacks. The resulting files are huge and unreadable, defeating the benefit of having a human-readable config. For example, a native xgboost JSON export contains a massive string representing all of its trees:\n{ \u0026#34;model_params\u0026#34;: { \u0026#34;type\u0026#34;: \u0026#34;xgboost.XGBClassifier\u0026#34;, \u0026#34;raw_model\u0026#34;: \u0026#34;{\\\u0026#34;learner\\\u0026#34;:{\\\u0026#34;gradient_booster\\\u0026#34;:{\\\u0026#34;model\\\u0026#34;:{\\\u0026#34;trees\\\u0026#34;:[{\\\u0026#34;id\\\u0026#34;:0,\\\u0026#34;split_indices\\\u0026#34;:[1,0,0],\\\u0026#34;split_conditions\\\u0026#34;:[0.5,1.1,-0.2],\\\u0026#34;left_children\\\u0026#34;:[1,-1,-1],\\\u0026#34;right_children\\\u0026#34;:[2,-1,-1]...[hundreds of thousands of characters truncated]...\\\u0026#34;\u0026#34; } } Instead of using this native text format, skeights extracts the tree parameters (such as split features, thresholds, child pointers, and leaf values) and saves them as typed numpy arrays in the safetensors file. Only the high-level scalar configuration remains in the JSON:\n{ \u0026#34;model_params\u0026#34;: { \u0026#34;type\u0026#34;: \u0026#34;xgboost.XGBClassifier\u0026#34;, \u0026#34;objective\u0026#34;: \u0026#34;binary:logistic\u0026#34;, \u0026#34;num_class\u0026#34;: 1, \u0026#34;feature_names\u0026#34;: [\u0026#34;f0\u0026#34;, \u0026#34;f1\u0026#34;, \u0026#34;f2\u0026#34;] } } The corresponding safetensors file contains the arrays:\nmodel/trees/split_indices (1200,) int32 model/trees/split_conditions (1200,) float32 model/trees/left_children (1200,) int32 model/trees/right_children (1200,) int32 This reduces file sizes significantly compared to native formats or pickle. For a model with 100 trees and 20 features:\nxgboost: 85% total reduction, with the JSON dropping from 1.8MB to 19KB. lightgbm: 30-43% total reduction, with the JSON dropping from 129-193KB to 6KB. The native format is still available as a fallback by passing format=\u0026quot;native\u0026quot;, and legacy artifacts load transparently.\nSecurity # By using JSON and safetensors, skeights avoids the arbitrary code execution risks of pickle out of the box.\nHowever, since the loader still has to instantiate Python classes specified in the JSON config, a crafted JSON file could try to import arbitrary modules. To prevent this, the loader restricts class imports to an allowlist containing only sklearn, lightgbm, and xgboost modules. Attempting to load a file with unauthorized module paths raises an error, strengthening the overall security model.\nDrawbacks # We have to add support for each estimator by hand, so we only cover the ones we\u0026rsquo;ve done so far. Backwards compatibility is also tricky as the library grows. We have tests that aim to catch breakages, but it isn\u0026rsquo;t guaranteed; then again, neither pickle nor skops guarantee this either.\nTry it out # We\u0026rsquo;re now using skeights in production. It\u0026rsquo;s MIT licensed and you can install it with pip install skeights. Contributions welcome, especially to widen the models we support.\nPronounced \u0026ldquo;skates\u0026rdquo;.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"3 July 2026","externalUrl":null,"permalink":"/projects/skeights/","section":"Projects","summary":"Extracting and open-sourcing our sklearn serialization library.","title":"skeights","type":"projects"},{"content":"","date":"3 July 2026","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"","date":"17 April 2026","externalUrl":null,"permalink":"/tags/agents/","section":"Tags","summary":"","title":"Agents","type":"tags"},{"content":"I\u0026rsquo;ve written before about how much I like devcontainers; they make your environment completely reproducible and keep projects isolated from each other.\nHowever, Claude has entered the picture, and doesn\u0026rsquo;t fit neatly into the model devcontainers were designed around.\nThe two-body problem # Devcontainers were built around one mental model: everything executes inside the container. VS Code attaches to it, your terminal runs inside it, your tools are installed inside it.\nClaude Code has the opposite assumption. It\u0026rsquo;s an agent running on your machine, shelling out commands directly. It expects to be able to reach your filesystem, your credentials, your Docker daemon. It was designed to operate alongside your environment, not inside it.\nThese two mental models conflict. Claude Code has no awareness of devcontainers: it won\u0026rsquo;t detect a .devcontainer in your repo, and it won\u0026rsquo;t use one if it exists. I\u0026rsquo;ve been experimenting with approaches to bridge that gap.\nOthers are reinventing the wheel # The community has noticed the problem, but I don\u0026rsquo;t think anyone has found the right solution yet. Kevin Scott first wrote up a DIY approach1 running Claude Code inside a heavily customised Docker container with pre-installed tooling, a dedicated GitHub identity, and tmux + git worktrees for parallelism.\nDocker Sandboxes2 later productised this idea, providing a microVM-based sandbox for Claude Code to run in. Both solve safety and isolation, but neither solves the reproducible environment problem. They ignore your devcontainer entirely and give you a fresh sandbox instead. Arcade did a hands-on review3 that captures the core issue well: setup is smooth for simple tasks, but falls apart fast with real workflows. For example, a sandbox restart wipes your Claude session. The irony is that both solutions are essentially hand-rolled devcontainers.\nFinding my own solution # Approach 1: install Claude inside the container # This was the obvious starting point. If Claude needs to run commands in the container, just put Claude in the container.\nThis was very easy to implement, but is more painful than it sounds:\nSession history breaks. Claude Code keys session history to the host path. If the path inside the container differs from the host (say, /home/alex/project vs /workspace/project), Claude doesn\u0026rsquo;t know they\u0026rsquo;re for the same project. A bind mount of .claude isn\u0026rsquo;t enough. Credentials don\u0026rsquo;t transfer. Currently you can\u0026rsquo;t pass Claude credentials from the host to the container like you can with Git credentials. You need to reauthenticate every time the container is rebuilt. It pollutes the project environment. Claude Code has nothing to do with your project\u0026rsquo;s actual dependencies, but it ends up baked into every contributor\u0026rsquo;s container. Not everyone uses Claude Code, and those who do shouldn\u0026rsquo;t need to maintain it as part of the project image. Docker access is limited. Running docker build from inside a container means Docker-in-Docker (fragile) or Docker-from-Docker (better, but needs to be set up). In a typical docker-compose setup (app, database, cache), Claude can only reach the container it\u0026rsquo;s running in. Claude on the host can reach all of them. Running Claude inside the container works, but it\u0026rsquo;s not a particularly smooth setup. It\u0026rsquo;s the wrong level of abstraction: the devcontainer should describe your project, not your AI assistant.\nApproach 2: get Claude on the host to use the container # The alternative is to keep Claude on the host but make it execute tasks inside the container\u0026rsquo;s environment. This immediately solves the issues with credentials, session history, and Docker access. The question is whether you can route Claude\u0026rsquo;s commands through the container from the host.\nIt turns out that Claude makes this quite easy with hooks, which let you intercept tool calls including bash commands. By prefixing every shell command with docker exec -it \u0026lt;container\u0026gt;, Claude runs its commands inside the container while staying on the host.\nI\u0026rsquo;m actually using this in practice. The repo below is a minimal working example of the hook approach.\nalxhslm/claude-devcontainer Shell 0 0 What surprised me was how seamlessly it works in practice. Because it\u0026rsquo;s a hook rather than a skill or prompt instruction, Claude cannot ignore it: every shell command is guaranteed to be routed through the container. This means Claude can interact with the container\u0026rsquo;s installed tooling (like running unit tests), while retaining access to general tools on the host like gh CLI or the Docker daemon.\nThe one genuinely unsolved piece is the container lifecycle. The devcontainer only spins up when VS Code opens it. I handled this through the CLAUDE.md file: Claude is instructed to check whether the container is running at the start of a session and ask the user to start it if not, but it\u0026rsquo;s a bit brittle. A native integration could handle this transparently as part of initialisation: detect the .devcontainer, spin it up, manage the lifecycle.\nThe case for native integration # The devcontainer spec was never really about VS Code. It\u0026rsquo;s a project-level environment definition. It captures everything your project needs to run: the base image, installed tools, environment variables, port mappings, lifecycle hooks.\nIn my opinion, that definition applies equally well to a human developer and an AI agent. Right now, VS Code is the primary consumer and Claude Code isn\u0026rsquo;t. However, if it were, the model becomes quite simple:\nhuman opens VS Code → devcontainer spins up; Claude Code runs → detects .devcontainer → uses the same container. One spec, two consumers. No duplication, no drift, no manual configuration.\nThe containerisation problem is already solved. The only missing piece is the link from Claude to the container.\nCloud environments come for free # The previous argument is about convenience. This one makes native devcontainer support feel necessary rather than nice-to-have.\nIf you want Claude Code to run autonomously in the cloud, you need to define an execution environment. Right now that means a setup script (custom Docker images aren\u0026rsquo;t supported yet), with no relation to your devcontainer. Setup scripts are cached between sessions, which helps, but it\u0026rsquo;s a hacky version of what Docker layer caching already gives you for free with devcontainers.\nGitHub Codespaces already solved this problem for human developers. Once you define your .devcontainer.json, you can get a fully reproducible cloud environment in Codespaces with no extra configuration. With native devcontainer support, Claude Code cloud tasks would work the same way. Your .devcontainer.json is the cloud environment spec.\nSince devcontainers relies on Docker, container builds are fast due to caching, and your local and cloud environments stay in sync for both humans and agents.\nOne spec, four consumers: VS Code, Codespaces, Claude Code (local), Claude Code (cloud). This is Anthropic\u0026rsquo;s problem to fix # The fix is actually really simple. Claude Code just needs to detect a .devcontainer in the repo root and route execution through it. I\u0026rsquo;ve already demonstrated it can work in principle. The only extra challenge is managing the container lifecycle, but that\u0026rsquo;s trivial for a company like Anthropic.\nThey\u0026rsquo;ve already solved the \u0026ldquo;how do tools talk to agents\u0026rdquo; problem with MCP. Right now, everyone building Claude Code workflows with containers is fumbling toward their own solution, but the devcontainer spec already exists and is widely adopted. Claude Code just needs to consume it.\nFor now, I\u0026rsquo;m using the command prefix approach locally which mostly works. However, the right answer is for Anthropic to add native devcontainer support. I\u0026rsquo;ve added this Claude Code issue on GitHub4. If you agree, please show your support by adding a comment to the issue!\nKevin Scott\u0026rsquo;s extensive write-up of his DIY Claude Code sandbox\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nSee this announcement on Docker\u0026rsquo;s blog.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nArcade did a hands-on review of Docker Sandboxes\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nGitHub issue asking for native devcontainer support for Claude Code\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"17 April 2026","externalUrl":null,"permalink":"/posts/claude_devcontainers/","section":"Posts","summary":"","title":"Claude should make friends with devcontainers","type":"posts"},{"content":"","date":"17 April 2026","externalUrl":null,"permalink":"/tags/devops/","section":"Tags","summary":"","title":"Devops","type":"tags"},{"content":"Thoughts on machine learning, engineering, and going down the occasional rabbit hole. I\u0026rsquo;m particularly interested in how new tools and techniques can help us all work more effectively.\n","date":"17 April 2026","externalUrl":null,"permalink":"/posts/","section":"Posts","summary":"","title":"Posts","type":"posts"},{"content":"","date":"17 April 2026","externalUrl":null,"permalink":"/tags/tooling/","section":"Tags","summary":"","title":"Tooling","type":"tags"},{"content":"","date":"13 April 2026","externalUrl":null,"permalink":"/tags/homelab/","section":"Tags","summary":"","title":"Homelab","type":"tags"},{"content":"I\u0026rsquo;ve been setting up a homelab using Runtipi. This is a self-hosted app platform which makes it easy to spin up containerised services using Docker Compose. The UI is simple and I\u0026rsquo;m already comfortable with containers, so it was a natural fit.\nOne of the best features is that you can create your own custom app store to host apps that aren\u0026rsquo;t in the official catalogue. I put together my own store for a handful of apps I couldn\u0026rsquo;t find elsewhere. I\u0026rsquo;ve focussed mostly on apps which make my homelab play nicely with the Apple ecosystem so far.\nalxhslm/runtipi-appstore My personal Runtipi app store for my homelab TypeScript 0 0 ","date":"13 April 2026","externalUrl":null,"permalink":"/projects/runtipi-appstore/","section":"Projects","summary":"","title":"Runtipi App Store","type":"projects"},{"content":"I recently set up an SMB share for Time Machine backups on my homelab. It\u0026rsquo;s a fiddly task fighting networking and permissions issues, and Docker configuration. I thought an AI agent would help speed things up.\nIt turned out to echo something I keep running into at work with ML models: to get the best out of an agent, you need to relinquish control in ways that feel uncomfortable, but only with the right guardrails in place.\nSetting up Time Machine over SMB # I wanted to set up an SMB network share on my Ubuntu homelab to use as a Time Machine backup destination1. The requirements were:\nContainerised within Docker Configurable enough to build a runtipi app later Modern Samba version (more efficient streaming for virtual APFS) Definition of done: The Mac can see the SMB share AND successfully read and write to it.\nThe naive approach: using myself as a middleman # Initially, I ran Claude Code on the homelab server and told it to set up the share. Then I\u0026rsquo;d manually test the connection from my Mac and report back any errors.\nThis was slow. I had to:\nDescribe what wasn\u0026rsquo;t working with enough context Remember to test all functionality (I kept forgetting to check write access) Go back and forth between machines I knew the agent could test the connection itself, and that it could SSH between the two machines. But I kept doing it manually anyway. Partly because I\u0026rsquo;m used to being hands-on, but more fundamentally, it felt risky to give an agent full SSH access to both machines for something this simple; like using a sledgehammer to crack a nut.\nThe shift: let the agent own everything # Then I realised: why am I doing half the work when the agent can do all of it?\nI changed the workflow:\nRun Claude Code on my Mac (not the server) Have it SSH into the homelab server Now it can configure the server AND test the connection from the Mac It sees full error context from both sides and can iterate on its own The agent would:\nSSH in, adjust the smb.conf Test the mount from the Mac side See the error message SSH back in, fix the config Repeat until it worked Then I just watched it debug itself. It was much faster.\nHowever, this only worked because I had tight constraints beforehand. I\u0026rsquo;d explicitly told it: modern Samba, must be containerised, needs to work with runtipi. Without those guardrails, it would have taken shortcuts that technically worked but didn\u0026rsquo;t meet my actual requirements.\nBuilding models is similar # At work, I\u0026rsquo;ve started to use AI agents to iterate on machine learning models for time-series forecasting2, and I\u0026rsquo;ve noticed the same pattern.\nIf you give the agent free rein to optimise a forecasting model without a well-defined evaluation methodology, it will confidently report unbelievable metrics. This is because the agent can cheat.\nData leakage in time-series is a classic failure mode. The model uses future data to predict the past, results look amazing, but they\u0026rsquo;re completely useless. The agent doesn\u0026rsquo;t know this is wrong because it just sees the metrics improving.\nWhat I\u0026rsquo;ve learnt: Unrestricted freedom without guardrails can give bad results. For ML work, the guardrail is evaluation. Proper train/test splits, validation that catches temporal leakage, metrics that actually measure what you care about. Once you\u0026rsquo;ve defined that, the agent can iterate through dozens of model configurations in minutes.\nFor sysadmin work, the guardrail is end-to-end testing. Not \u0026ldquo;config looks right\u0026rdquo;, but \u0026ldquo;does it actually work when I try to use it?\u0026rdquo;\nThe pattern # I think relinquishing control is psychologically hard, especially for technical people who are used to being hands-on. But the agent is 10x faster at iteration than we\u0026rsquo;ll ever be.\nThe key is knowing where our judgement still matters:\nLess valuable: Knowing specific smb.conf or PyTorch syntax, because the agent can do this well already. More valuable: Defining what \u0026ldquo;correct\u0026rdquo; looks like, and how to test for it, because this is where the agent can trip up. The paradox I\u0026rsquo;ve found: I need to be more rigorous about defining success precisely so I can be hands-off about how it gets achieved. For me, this has meant:\nDefining requirements clearly beforehand: otherwise the agent takes shortcuts around them Making \u0026ldquo;done\u0026rdquo; testable: giving the agent something it can verify itself Giving it access to test its own work: getting myself out of the iteration loop Intervening when it gets stuck: asking it to summarise root cause and suggest alternatives Where I think domain expertise matters # I think that our role as engineers has shifted; it\u0026rsquo;s more about setting up the problem so the agent can explore it effectively, rather than doing the manual iteration ourselves.\nFor time-series forecasting: \u0026ldquo;Does this evaluation catch the ways my model could be wrong?\u0026rdquo;\nFor SMB configuration: \u0026ldquo;Does this test prove the system actually works end-to-end?\u0026rdquo;\nFor any technical work: \u0026ldquo;What are the failure modes I care about, and how do I detect them?\u0026rdquo;\nIf you define it correctly, the agent can move fast. If not, we\u0026rsquo;re just automating the production of plausible-looking slop.\nI\u0026rsquo;m still not entirely sure what the best way to enforce these guardrails is. Is it through writing skills that the agent can reference? Is it just careful prompting? I suspect it\u0026rsquo;s a mix of both, but I haven\u0026rsquo;t figured out the right balance yet.\nI also wrote about my issues with Gemini on this task.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nInspired by Karpathy\u0026rsquo;s autoresearch.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"12 April 2026","externalUrl":null,"permalink":"/posts/ai_agents_beyond_coding/","section":"Posts","summary":"","title":"Learning to Let Go: Using AI Agents Beyond Coding","type":"posts"},{"content":"I recently set up an old Dell PC running Ubuntu server as my homelab. It threw up the usual networking and permissions issues, and I thought an AI coding agent would massively speed up the debugging. I\u0026rsquo;ll walk through one task as an example (setting up a Samba share for Time Machine backups), which ended up highlighting the difference in utility between the agents pretty starkly.\nClaude Code is £18/month. For hobby use, that felt expensive. I assumed Gemini would work just as well for less money. I had access via a Google AI trial, so why not try it?\nSpoiler: I was wrong on multiple levels.\nAttempt 1: Gemini CLI works fine\u0026hellip; for simple tasks # To be fair, raw Gemini CLI actually works well for straightforward conversions. When I needed to convert my LaTeX CV to Typst, it handled it without issues.\nI thought: \u0026ldquo;If it can handle that, homelab debugging should be fine too.\u0026rdquo;\nThe real test came when I tried to set up a Samba share for Time Machine backups. This felt like a fairly easy, well-scoped task; find a Docker image that meets two requirements:\nModern Samba version (more efficient streaming for virtual APFS) Configurable Docker image (to build a runtipi app later) and then sort out any networking or permissions issues. Gemini found two Docker images:\ndperson/samba (old version) crazy-max/docker-samba (new, but hard to configure) I explicitly told it: \u0026ldquo;I need modern Samba.\u0026rdquo; This clearly rules out dperson.\nIt kept suggesting dperson anyway. Over and over. Completely ignored my constraint.\nIt also kept getting stuck in infinite loops, where I had to intervene.\nAttempt 2: Adding a harness didn\u0026rsquo;t help # I thought that this was because the Gemini CLI harness wasn\u0026rsquo;t very good. So I tried using OpenCode (an agentic harness) and configured it to use Gemini under the hood.\nWhat initially drew me to it was that it\u0026rsquo;s a genuinely sophisticated product. Unlike raw CLI tools, it has a web GUI so you can code from the browser without SSH, a dedicated thinking mode, and you can even configure different models for different roles; a cheaper model for small edits, a more powerful one for planning for example. On paper, it seemed like a step up in every way.\nNew harness, same results # Unfortunately, despite the fancy UI, the experience was very similar to Gemini CLI. It kept falling back to dperson/samba, which didn\u0026rsquo;t meet my initial requirements. At least it didn\u0026rsquo;t loop forever — it gave up and explained why it was stuck.\nHere\u0026rsquo;s the key insight: adding a more sophisticated harness made no difference. That doesn\u0026rsquo;t mean the harness is irrelevant; it might well be making things worse in ways that are hard to observe. But it\u0026rsquo;s clearly not enough to compensate for a weak underlying model.\nThe API cost shock # I was using the cheaper Gemini Flash as the implementation agent, but I thought it might not be good enough for this task. So I switched over to using the more expensive Gemini Pro via API.\nI spent roughly £5 in one evening across various homelab debugging tasks.\nThat\u0026rsquo;s 25% of a monthly Claude Code subscription. In one evening. And my Samba server still didn\u0026rsquo;t work.\nAttempt 3: Claude Code actually solved it # At this point, I realised I should probably try what I know works in my day job: Claude Code.\nClaude\u0026rsquo;s approach to the same Samba problem:\nPlanned before jumping in; actually understood what I was trying to achieve Recognised that the two images I\u0026rsquo;d found weren\u0026rsquo;t a good fit, and went looking for alternatives Found mbentley/docker-timemachine which is a purpose-built image I hadn\u0026rsquo;t come across SSH\u0026rsquo;d into the homelab and tested from both sides Actually solved it The rule of thumb is that for low usage, PAYG is cheaper. So I initially tried Claude via API, thinking it would be cheaper than the subscription. While I did solve my problem, I spent another ~£5 in the process.\nAfter this result, I bit the bullet on Claude Pro (£18/month). It\u0026rsquo;s a proper product rather than just API access; some features are paywalled even with an API key, such as remote control, which lets you kick off tasks from your phone while away from your desk.\nThe three lessons # Lesson 1: A better harness can\u0026rsquo;t fix a weaker model # Simple agents like Gemini CLI are fine for simple conversions like LaTeX to Typst. However, they struggle severely when solving more complex tasks.\nThere\u0026rsquo;s clearly a fundamental model limitation, and not just a tooling issue. Gemini kept taking shortcuts and ignoring my constraints, even with a better agentic harness.\nLesson 2: Claude still has an advantage right now # Right now, you can\u0026rsquo;t get the same experience for less money. The quality gap is real: it\u0026rsquo;s not just about features, it\u0026rsquo;s about problem understanding, reasoning, and finding the right solutions.\nThat said, this space is moving fast. OpenCode with a stronger model choice might close the gap; I only tested it with Gemini. And the model landscape in 6 months will look very different. It\u0026rsquo;s worth keeping an eye on.\nLesson 3: The subscription threshold is lower than you think # This goes against the conventional rule of thumb that API pricing is cheaper for light usage.\nI spent £5 in one evening of hobbyist debugging. At that rate, the £18/month subscription pays for itself after just 4 evenings of work — and that\u0026rsquo;s not counting the time lost fighting with Gemini.\nDeveloper time is expensive (even if it is just for your hobbies!). The subscription makes sense much earlier than I\u0026rsquo;d expected.\n","date":"27 March 2026","externalUrl":null,"permalink":"/posts/claude_code_vs_gemini/","section":"Posts","summary":"","title":"Tried to save £18/month on Claude Code, wasted £5 in one evening on Gemini","type":"posts"},{"content":" The trigger: a new TeX Live release # I used LaTeX at university and loved how it let me focus on the meaning of my writing rather than fiddling with formatting. That principle still holds. But my software practices have grown since then; I\u0026rsquo;ve come to rely on things like formatters, linters, and proper package management, and LaTeX\u0026rsquo;s equivalents don\u0026rsquo;t really measure up.\nWhen I went to update my CV recently (just keeping it current, not job hunting!), my LaTeX devcontainer had broken. I\u0026rsquo;d previously written about setting up CI for a LaTeX CV, so this was a setup I\u0026rsquo;d invested real effort into. TeX Live cuts a new annual release and drops the old one from mirrors; my devcontainer was still trying to download the 2025 release, which was gone.\nRather than debug it, I decided to try Typst, a modern alternative I\u0026rsquo;d seen on my GitHub feed months ago. I was trialling Gemini CLI at the time, so I gave it this task: convert my LaTeX CV to Typst. I iterated on the output. The whole thing took 30 minutes. Within an hour of starting, I was wondering why I\u0026rsquo;d spent so much time working around LaTeX\u0026rsquo;s limitations.\nWhy Typst is better # Compilation is instant # My CV was taking around 10 seconds to compile in LaTeX; not terrible, but enough to break flow. Typst is fast enough that you don\u0026rsquo;t notice it. The live preview actually works, and iterating on formatting feels completely different when the feedback is immediate.\nThe tooling is modern # LaTeX does have language servers, formatters, and linters, but they\u0026rsquo;re niche and under-invested. Most LaTeX users are academics who don\u0026rsquo;t care about this stuff, so the ecosystem reflects that. latexindent is a Perl script maintained by one person; language server support exists but is nowhere near the quality you\u0026rsquo;d get with a mainstream programming language.\nTypst is built for people who do care. LSP support, linters, and helpful error messages are first-class. It feels like a tool built in the 2020s.\nPackage management actually works # There\u0026rsquo;s no way to pin package versions in LaTeX. Your document might break when packages update and you have no way to prevent it. I hacked together a workaround using a requirements.txt file, but even that doesn\u0026rsquo;t support version pinning.\nTypst has a proper dependency model with versioning. Reproducible builds work without custom hacks.\nOne compile pass # Want citations in LaTeX? You\u0026rsquo;re compiling at least twice: once for the document structure, once to resolve references. Typst handles everything in a single pass.\nThe broader point # LaTeX became the default because academics learned it at university, taught it to the next generation, and journals standardised on it. That\u0026rsquo;s down to network effects, not merit. For journal submissions or collaboration with LaTeX-only colleagues, you may not have a choice. For everything else, it\u0026rsquo;s worth asking whether it\u0026rsquo;s still the right tool.\nWhat made it easy to ask that question here was that AI tools have collapsed the switching cost. What used to mean weeks of manual rewriting now takes 30 minutes. The economics of trying something better have changed completely.\nI saw Typst on my GitHub feed months before I tried it. The devcontainer breaking was the nudge I needed. If you\u0026rsquo;re hitting friction with an incumbent tool, the bar for trying an alternative is lower than it used to be; it\u0026rsquo;s worth a try.\n","date":"8 March 2026","externalUrl":null,"permalink":"/posts/typst/","section":"Posts","summary":"","title":"My LaTeX devcontainer broke. So I tried Typst instead.","type":"posts"},{"content":"","date":"8 March 2026","externalUrl":null,"permalink":"/tags/typesetting/","section":"Tags","summary":"","title":"Typesetting","type":"tags"},{"content":"","date":"21 December 2025","externalUrl":null,"permalink":"/tags/productivity/","section":"Tags","summary":"","title":"Productivity","type":"tags"},{"content":"","date":"21 December 2025","externalUrl":null,"permalink":"/tags/review/","section":"Tags","summary":"","title":"Review","type":"tags"},{"content":"In principle, a calendar app is nothing more than a digital list of events. It should be simple, yet in our age of digital noise and constant distraction, it becomes the our most important tool for managing our time 1.\nI rely on recording everything, especially at work when I’m deep in concentration. My particular struggle is with events about a week or so in the future: too far to hold in memory, but not distant enough to be reliably pre-emptied by colleagues. For the habitually forgetful like me, a calendar app is not just a convenience; it\u0026rsquo;s an essential external brain.\nWhile simple tasks like maintaining work-life balance and remembering anniversaries can be handled by a paper diary, the demands of the modern work week require more opinionated, thoughtfully designed software. This is where apps truly shine, offering features like:\nDaily Time Blocking: Planning your day hour-by-hour, potentially overlaying your tasks with your meetings. Intelligent Reminders: Telling you precisely when to leave, factoring in travel time. Video Call Integration: Quick links to join meetings instantly. I was a long-time Fantastical 2 user, and was lucky enough to be grandfathered into some paid features of Fantastical 3. However, the constant pop-ups advertising premium-tier features eventually broke my resolve 2, which led me to reassess the current landscape.\nWhat do I want from a calendar app? # Before diving in, let me set our my criteria. I specifically ruled out AI scheduling assistants (like Motion), as scheduling itself isn\u0026rsquo;t my core problem. Instead, I care about the following:\nMultiple Account Support: Seamlessly supporting both work and personal lives, as the two inevitably interact. Task Integration: Ability to show tasks (like Apple Reminders or Google Tasks) alongside events for better daily planning. Meetings: Easy ways to see upcoming meetings and links to quickly join video calls. Note that I focussed only on apps for MacOS since this is what I use daily. I didn’t consider windows or android apps (although some of the options below are available on these platforms). At the same time, I want to be able to have the same interface on my iPhone, so I only consider apps available on iOS as well.\nWhat options are there? # Apple Calendar # Apple Calendar is installed by default on all Apple devices, so is the easiest option. It\u0026rsquo;s perfectly functional, but relatively bare bones.\nPros Cons Free and deeply integrated with OS. Forces reliance on Apple services (Apple Maps, Apple Reminders). Native apps on all Apple devices, including widgets. No-frills experience—does the fundamentals but lacks desktop features like a menu bar calendar. Can show Reminders alongside your calendar since iOS 18/MacOS Seqioua. Can require additional third-party apps (e.g. Itsycal for a menubar calendar, Meeting bar for meeting links, or Dato which can do both) to extend functionality. Time to Leave notifications using its native knowledge of your location. Does show events on desktop widget, but this obviously usually hidden Google Calendar # Google Calender needs no introduction. Purely as a tool for managing your events, it is very good. However, lack of multi-user support is a deal-breaker for me.\nPros Cons Free on all platforms Have to manage different Google accounts separately. However, you can sync a personal iCal feed (read-only) to your work calendar and vice versa. Personally I prefer to keep my work and personal accounts separate. Supports proprietary functionality for Google accounts (e.g., showing your working location). On desktop, it’s just a browser tab, so no rich features such as meeting links or menubar access. Rich features on mobile apps (e.g., widgets) Google calendar app Notion Calendar # Notion Calendar was formerly known as Cron before being bought by Notion. Unfortunately, development appears to have stagnated since then. For example, there is still no week view on the mobile app!\nPros Cons Free with Notion. Can only display tasks from Notion databases, limiting utility for those who use other task managers. Great integration with any Notion database (projects, day-to-day tasks etc). Mobile app lacks feature parity - it doesn’t even have a week view yet! Includes a decent menubar calendar on Mac and handy conference call notifications. Fantastical # Fantastical is widely regarded as the best calendar app for Apple devices. However, they recently switched pricing models to an expensive subscription which has annoyed a lot of users.\nPros Cons Packed with rich features like the menu bar calendar and instant meeting links. Expensive subscription if you want all the premium features, though it is high-quality software. Excellent design, particularly the Schedule view. I don’t know why other apps haven’t copied this fluid timeline. Syncs with many different services such as Google Tasks, Apple Reminders, or Todoist. The free tier still has a lot of features, making it highly usable without paying. BusyCal # BusyCal is the main competitor to Fantastical. It has an up-front pricing model (with a limited widow of updates), but doesn’t quite have the same polish as Fantastical.\nPros Cons One-off purchase for bioth the desktop and mobile apps, avoiding subscriptions. MacOS licence only includes 18 months of free updates 3. Excellent customisation: includes Smart Filters, Tags, and highly configurable views. The visual design is more functional and less aesthetically polished than Fantastical. Deep task integration, supporting multiple external task managers. iOS app is a separate purchase (though also one-off). Includes unique features like integrated weather forecasts, moon phases, and graphics. Why are there so few native calendar apps? # One of my biggest frustrations during this search was realising just how many modern calendar tools are moving towards being purely web-based; the obvious example being Google calendar, but Notion calendar is just an electron wrapper. The situation is even worse if you’re not on MacOS. Unfortantely, it seems that calendar apps aren’t particularly profitable, so few developers are willing to build good calendar apps. If you’re a developer, it makes much more sense to focus on web apps since it’s cross-platform and maximises your potential user-base.\nHowever, a web app is \u0026ldquo;trapped\u0026rdquo; in the browser so is inherentely limited. It cannot easily \u0026ldquo;leave\u0026rdquo; its tab to provide the deep system integration that makes a calendar most useful. For example, you can’t have things like widgets, or have to rely on less robust browser notifications. Multi-user support is also often lacking; even in 2025, the web version Google Calendar can only show you one calendar account! Compare this to the mobile app which can support multiple accounts, and has native widgets.\nSticking to what you know # So, after this deep dive, what did I go for?\nIf you’re already embedded in the Notion ecosystem, Notion Calendar provides good task integration and a decent desktop app, all for free - although the mobile app is pretty poor. If you are budget-conscious and willing to piece together features, Apple Calendar + third-party helper apps is a surprisingly capable option, albeit less polished.\nHowever, in my opinion, only Fantastical offers a seamless blend of multiple accounts, task display, and crucial quality-of-life features like menubar access and meeting join links. None of the alternatives quite matches its thoughtful design, especially the excellent Schedule view 4. It\u0026rsquo;s just as good on iOS too.\nI’ve decided to stop the endless search. While the subscription is a consideration, Fantastical still excels at my needs. Sometimes, the best solution is the one you already know. I just wish there were more options to choose from!\nYou can read more about this topic in Deep Work by Cal Newport.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nOf course there is no option to just hide the paid options 😫\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nYou keep the app forever, but pay a renewal fee for future feature/OS compatibility updates.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nI’m not the only person who came to this conclusion it seems.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"21 December 2025","externalUrl":null,"permalink":"/posts/calendar_apps/","section":"Posts","summary":"","title":"The Quest for the perfect MacOS Calendar App","type":"posts"},{"content":"","date":"18 March 2025","externalUrl":null,"permalink":"/tags/investing/","section":"Tags","summary":"","title":"Investing","type":"tags"},{"content":"For a long time, I put off sorting out my pension because the whole concept felt overwhelming. It sounded quite complicated, filled with technical jargon, and I wasn\u0026rsquo;t sure where to start. But as I dug into it, I discovered that a pension is really just a tax-efficient investment wrapper, and not something fundamentally different from investing in general. Once I understood that, it became much less intimidating. Once I chose a suitable (and cheap) SIPP platform 1, \u0026ldquo;all\u0026rdquo; I need to do was decide what to invest in.\nDiscovering rational investing # Initially, I assumed that successful investing meant picking individual stocks, keeping up with market trends, and making the right bets. I didn\u0026rsquo;t feel like I had any expertise in the world of finance, and it sounded like a full-time job.\nThen I read Investing Demystified by Lars Kroijer, which introduced me to a much simpler approach to investing requiring little effort. Instead of trying to beat the market, a rational investor takes a more passive approach by investing in a cheap global equity index fund to generate returns. Essentially you allow the market to choose the optimal portfolio of shares for you.\nYou can then balance this out with some government bonds as a minimal risk asset (which can also be cheaply invested in with an index fund), so you can tune your risk level. This passive approach takes the complexity out of investing, significantly reduces your costs, and gives you a solid foundation for long-term growth.\nBuilding a portfolio simulation tool # Once I decided to go with passive investing, the next challenge was tuning the portfolio to match my risk level. The right balance between equities and bonds depends on your individual needs; eg time horizon, risk tolerance, and financial goals.\nI struggled to find any freely available tools to help with this, so decided to build my own. Using the model presented in Investing Demystified, I created a simple Streamlit dashboard to simulate portfolio growth over time.\nalxhslm/wealthypy Financial planning tool for rational investors Python 2 0 Features of the tool # Set a starting amount and monthly contributions, which can vary over time.\nAdjust the equity and bond returns, as well as the equity-bond split, which can also vary over time.\nSimulate how your portfolio might grow while accounting for equity volatility using a Monte Carlo-like approach.\nThis has helped me choose how much to contribute to my pension, fine-tune the portfolio to my needs, as well as gain confidence in my overall financial plan. I hope it can help others do the same.\nAt some point I hope to further improve the tool to incorporate additional assets like corporate bonds (also recommended in the book) and evaluate your investment strategy on historical data as well.\nI would recommend looking at MSE as it can depends a lot on how much money you have\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"18 March 2025","externalUrl":null,"permalink":"/projects/wealthypy/","section":"Projects","summary":"","title":"Wealthypy","type":"projects"},{"content":"","date":"6 March 2025","externalUrl":null,"permalink":"/tags/hardware/","section":"Tags","summary":"","title":"Hardware","type":"tags"},{"content":"I’ve been using a Logitech MX Keys for a while now, and while it’s a solid keyboard, it’s a bit on the large side. I wanted something more compact, with a better typing experience—so I started looking into mechanical keyboards.\nWhat I Wanted in a Keyboard # I had a few key requirements for my next keyboard:\nWireless – I like a clean desk setup with minimal cables. Plus, having the option to use it with an iPad in the future is a nice bonus. TKL or 75% Layout – I prefer a more compact design so I have more space for my mouse. Function Keys – I use these a lot, especially for IDE shortcuts, so they were a must-have. Tactile Switches – This keyboard is for typing, not gaming. I wanted something that felt great to type on. UK ISO Layout – I don’t like switching between layouts, and since my MacBook uses ISO, I wanted my external keyboard to match. Mac Keycaps – I pretty much exclusively use Macs now, so I didn’t want Windows key labels. Professional Look – Nothing too flashy or “gamer-y.” Just a clean, smart design. Why I Chose the Keychron V1 Max # Keychron is one of the few manufacturers that actually make ISO layout mechanical keyboards, so they were the obvious brand to look at. Initially, I considered the K2 Pro, but then I came across the V1 Max. It was widely recommended as one of the best budget options and seemed to punch well above its price point.\nIt brings in features from Keychron’s higher-end Q1 Pro, which made it even more appealing:\nVolume knob – Adjusting volume with dedicated keys takes too long, so this was a great addition. 2.4GHz Wireless – Bluetooth can be unreliable, so having a more stable wireless option was a big plus. VIA \u0026amp; Keychron Launcher Support – This meant I could easily remap keys to suit my needs. A great typing experience # The Banana Switches on this keyboard are excellent. They offer great tactility with a satisfying “thock,” but they’re not overly loud, making them a significant improvement over the Browns I’ve used before.\nThe overall build quality is impressive. Despite being plastic, it looks and feels far more expensive than it actually is. One nice touch is that all the screws are hidden on the bottom, unlike the regular V1, which contributes to a cleaner design.\nThe wireless connection has been reliable, pairing quickly over Bluetooth and maintaining a stable connection to my M3 MacBook Pro for work. However, my old 2015 MacBook Pro struggled with connectivity, although I suspect that’s more down to the laptop than the keyboard itself.\nBattery life is another strong point. It lasts a long time between charges, and macOS allows you to check the battery level easily.\nI wasn’t particularly interested in backlighting, but I have to admit it looks good. The RGB isn’t something I’d have sought out, but it adds a nice touch to the overall aesthetic.\nTerrible customer service # Before buying, I saw a lot of Reddit posts complaining about Keychron’s poor customer service. Many people recommended buying from a big reseller to make returns easier. I didn’t take this too seriously—I figured the risk of a keyboard arriving faulty was low, and I got a discount from Keychron directly, making it much cheaper than elsewhere.\nWell, lesson learned.\nThe keyboard arrived non-functional. It wouldn’t hold a charge and only worked when plugged in. On top of that, the keys weren’t mapped correctly—it was stuck in Windows mode, with the Command and Option keys swapped around.\nI tried updating the firmware via Keychron Launcher, but that didn’t help. I emailed Keychron… and got no reply.\nAfter doing some research, I found that loose cables inside the keyboard were a common issue. I opened mine up and saw the ribbon connector between the daughterboard 1 was actually broken, meaning the ribbon cable wouldn’t stay in place. This explained both why the keyboard wouldn\u0026rsquo;t charge, and why it was stuck in Windows mode. I assumed I’d need a new PCB and waited to hear back from Keychron.\nA week passed, and I still hadn’t heard anything, so I got frustrated and decided to fix it myself. A bit of electrical tape and superglue later, the ribbon was secured. I added extra tape to prevent future issues, put everything back together, and—finally—the keyboard was working perfectly.\nMy slightly dodgy repair to the ribbon connector Thankfully, it’s a product designed to be opened up, so it was a relatively easy fix. But I really shouldn’t have had to do this.\nFinal Thoughts # Despite the issues, I’d still recommend the Keychron V1 Max—because it’s genuinely a great keyboard. However, it’s clear that Keychron has serious quality control and customer service problems. If you’re going to buy one, do yourself a favour and order from a reputable retailer, even if it costs a little more. It’ll likely arrive quicker, and you’ll have an easier time if anything goes wrong.\nThe only danger now is that I’ve caught the mechanical keyboard bug. I’m already eyeing up an upgrade to something with a metal build… Not a cheap hobby!\nThis is what the Mac/Windows and BT/2.4GHz/Wired switches are mounted to, as well as the USB-C port for data and charging\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"6 March 2025","externalUrl":null,"permalink":"/posts/keychron_v1_max/","section":"Posts","summary":"","title":"My Mechanical Keyboard Journey: Keychron V1 Max Review","type":"posts"},{"content":"","date":"15 February 2025","externalUrl":null,"permalink":"/tags/fitness/","section":"Tags","summary":"","title":"Fitness","type":"tags"},{"content":" Why I Joined F45 # I recently joined F45 to lose some weight and force myself to do more weight training. I generally prefer running or cycling outside, so this seemed like a good way to mix things up.\nF45 promotes its LionHeart heart rate monitor, which connects to a receiver in the studio and provides:\nLive heart rate data displayed on screens during the class Calories burned and LionHeart points (whatever those are) Since I already use a Garmin Forerunner running watch, I wasn’t interested in paying extra for a proprietary device that offers minimal added benefit. However, I soon realised there was more to the system than I initially thought.\nThe Strava Sync Let-down # F45 allows you to link your account to Strava, automatically posting your workout along with the type of class it was. In theory, this should provide a more accurate calorie estimate because F45 knows what you did in class.\nAfter linking my F45 account, I went to my first session expecting my heart rate data to sync to Strava. But when I checked the activity later - nothing. No HR data. 😢\nThe Reddit Discovery: ANT+ Compatibility # A quick Google search revealed that LionHeart data is supposed to sync to Strava. Meaning, if I had a LionHeart monitor, my heart rate and calorie data would show up.\nDigging deeper, I found a Reddit post that revealed an interesting detail: LionHeart just uses ANT+. That means any ANT+ heart rate monitor - like my Garmin Dual HRM chest strap or my Garmin watch - should work. 🎉\nSetting Up My Garmin HRM With F45 # I chose to use my Garmin HRM over my running watch as it is typically more accurate, especially when you’re moving your arms a lot like you do at F45.\nF45 requires you to enter a Device ID in the app to link your heart rate monitor, which corresponds to the ANT+ Sensor ID. To find the Sensor ID of my Garmin HRM, I checked my Garmin Edge 530 which was already paired to it, which reported a value of 863436. I entered this into the F45 app and eagerly awaited my data appearing on the class screens.\nUnfortunately, it didn’t work as expected. When I went to my next class, a device showed up on the screen as Guest#11486 but wasn’t linked to my account. 🤔\nThe “Magic Number” Problem # Another Reddit post mentioned a “magic number” that needed to be entered instead. This corresponds to the number after Guest#. But where did this number come from?\nThe issue stems from how ANT+ assigns device addresses. Originally, ANT+ used 16-bit addresses, but newer devices use 20-bit addresses. Some systems (including F45) still expect a 16-bit number, meaning we need to convert our 20-bit Sensor ID to its 16-bit equivalent.\nConverting an ANT+ Sensor ID for F45 # To calculate the correct Device ID for F45:\nConvert the number from your Garmin device to Hexadecimal. Example: 863436 → 0xD2CCC Drop the first 4 bits (the first hexadecimal character). Example: 0xD2CCC → 0x2CCC Convert back to Decimal. Example: 0x2CCC → 11486 Once I entered 11486 into the F45 app, my Garmin HRM was recognised when I to my next class, and my heart rate data was displayed correctly on the screens. 🎯\nFinal Thoughts # It’s always satisfying to figure out why something works rather than just following a vague fix. If you already own an ANT+ heart rate monitor, there’s no need to buy an F45 LionHeart — you can simply use this method to get it working.\nHopefully, this helps others looking to use their own devices with F45!\n","date":"15 February 2025","externalUrl":null,"permalink":"/posts/f45_lionheart/","section":"Posts","summary":"","title":"Hacking F45 LionHeart: Using a Garmin HRM Instead","type":"posts"},{"content":"I have recently started to use LaTeX again to write my CV after a long hiatus. A lot has changed since I wrote my PhD thesis using LaTeX back in 2020, both in terms of the available tooling and my knowledge of modern software development practices. Therefore I made some significant updates to my LaTeX workflow which I will describe below.\nSwitching to VS code as an IDE # During my PhD, I primarily used the following LaTeX specific editors:\nTeXStudio on Windows Texifier (formerly TeXpad) on the Mac The main reasons why I chose to use a specific LaTeX IDE were:\nThey automatically work out the build process for you, so you can just press build and it spits out a PDF They provide LaTeX specific features such as assembling the document hierarchy for easy navigation That was what everyone else in my group did To be honest, I never really loved any of these options because they weren’t particularly good text editors with odd keybindings and limited theming options. With the exception of Texifier on the Mac, they tend to be ugly GTK apps and are not particularly enjoyable to use.\nThe obvious solution to this is to use the IDE I use everyday for Python development which is VS Code. To get LaTeX specific functionality like syntax highlighting, you can use the LaTeX Workshop extension. When I tried using this extension back in 2020, I found it to be lacking in features. However, it has matured considerably since then and IMO gives just as good an experience as a LaTeX specific editor.\nTo automatically work out the build process, you can use the latexmk Perl script in place of pdflatex etc. This gives you a simple just build my document command which you can use with any editor you like.\nPre-commit hooks # Spell check # Once you’ve written your document, you obviously want to check for spelling mistakes. When I wrote my PhD thesis, I did this completely manually which is clearly not a full proof strategy. I knew there must be some tool for this, and I found the cspell utility from the makers of the CSpell VS code extension.\nTo add a custom LaTeX dictionary for cspell so that it doesn\u0026rsquo;t flag macros, environments etc, you can add the following lines to the cspell.json configuration file:\n{ \u0026#34;import\u0026#34;: [\u0026#34;@cspell/dict-latex/cspell-ext.json\u0026#34;] } Since I wanted to perform these checks automatically, I configured a pre-commit hook to run on all LaTeX files by adding the following to .pre-commit-config.yaml:\nrepos: - repo: https://github.com/streetsidesoftware/cspell-cli rev: v8.11.0 hooks: - id: cspell entry: cspell-cli language: node types: [file] files: \\.(tex|sty|cls)$ Formatting # I’ve now become accustomed to formatting all of my Python code using black, because it improves legibility and removes whitespace changes from git diffs. Fortunately there is a similar formatter available for LaTeX in latexindent.pl. This can also be configured as a pre-commit hook as follows:\nrepos: - repo: https://github.com/cmhughes/latexindent.pl rev: V3.23.4 hooks: - id: latexindent entry: latexindent.pl args: [\u0026#34;-wd\u0026#34;, \u0026#34;-s\u0026#34;, \u0026#34;-c\u0026#34;, \u0026#34;.latexindent\u0026#34;] language: perl types: [file] files: \\.(tex|sty|cls)$ where the following flags have been set:\n-wd ensures it writes any changes to file -s suppresses any output -c sets the directory for temporary files (so that you can add it to .gitignore) Unfortunately, unlike black, this does not ship as a binary and requires perl to be installed on your system. I had to install some additional packages on my Mac to get this to work:\ncpan -i YAML::Tiny File::HomeDir Unicode::GCString A more robust solution is use a devcontainer as described below.\nUsing a devcontainer # I’m a strong advocate for using devcontainers to standardise your environment, and writing LaTeX documents is no different. There are many pre-built devcontainer definitions such as qdm12/latexdevcontainer or a-nau/latex-devcontainer, but I decided to write my own, because I wanted to a bit more control of the LaTeX installation. Once I had got this working reliably, I was able to make use of GitHub codespaces. This gave me something akin to Overleaf, but using the more familiar VS Code editor.\nInstalling LaTeX dependencies # To install LaTeX on a Debian-based OS, you need to install some additional system dependencies:\n# Install required system packages for LaTeX RUN apt-get update \\ \u0026amp;\u0026amp; apt-get install -y --no-install-recommends \\ ghostscript \\ gnupg \\ perl \\ \u0026amp;\u0026amp; rm -rf /var/lib/apt/lists/* To use latexindent.pl, you also need to install some additional perl packages:\n# Install dependencies needed by latexindent RUN apt-get update \\ \u0026amp;\u0026amp; apt-get install -y --no-install-recommends \\ libunicode-linebreak-perl\\ libyaml-tiny-perl \\ libfile-homedir-perl \\ \u0026amp;\u0026amp; rm -rf /var/lib/apt/lists/* Minimal LaTeX installation # The main challenge I ran into was configuring a minimal LaTeX installation, with only those packages required for my document. To install TeXLive with minimal packages you can use the following bash script:\n#!/bin/bash set -e echo \u0026#34;==\u0026gt; Install TeXLive\u0026#34; mkdir -p ./texlive MIRROR_URL=\u0026#34;$(curl -w \u0026#34;%{redirect_url}\u0026#34; -o /dev/null https://mirror.ctan.org/)\u0026#34; curl --output-dir ./texlive -OL \u0026#34;${MIRROR_URL}systems/texlive/tlnet/install-tl-unx.tar.gz\u0026#34; curl --output-dir ./texlive -OL \u0026#34;${MIRROR_URL}systems/texlive/tlnet/install-tl-unx.tar.gz.sha512\u0026#34; curl --output-dir ./texlive -OL \u0026#34;${MIRROR_URL}systems/texlive/tlnet/install-tl-unx.tar.gz.sha512.asc\u0026#34; mkdir -p ./texlive/installer tar --strip-components 1 -zxf ./texlive/install-tl-unx.tar.gz -C ./texlive/installer sudo ./texlive/installer/install-tl -profile=./texlive.profile echo \u0026#34;==\u0026gt; Clean up\u0026#34; rm -rf \\ /usr/local/texlive/texdir/install-tl \\ /usr/local/texlive/texdir/install-tl.log \\ ./texlive which can then be called within your Dockerfile. The texlive.profile file is used to configure what will be installed. Here is my version which only installs the default LaTeX packages, and no documentation or source code:\nselected_scheme scheme-infraonly TEXDIR /usr/local/texlive/ TEXMFCONFIG ~/.texlive/texmf-config TEXMFHOME ~/texmf TEXMFLOCAL /usr/local/texlive/texmf-local TEXMFSYSCONFIG /usr/local/texlive/texmf-config TEXMFSYSVAR /usr/local/texlive/texmf-var TEXMFVAR ~/.texlive/texmf-var option_doc 0 option_src 0 collection-latex 1 Unfortunately I couldn’t find a way to specify a set of LaTeX packages to be installed. As a workaround, I created a texlive-packages.txt file with a list of package names as follows:\ntabularx graphicsx ... To then install only these packages, you add the following to your post_start.sh file:\ncat texlive-packages.txt | sed -re \u0026#39;/^#/d\u0026#39; | xargs sudo tlmgr install which will install only the specified packaged using the tlmgr package manager. Obviously this does not pin the package versions, but at least you don\u0026rsquo;t have to install every LaTeX package under the sun for a simple document.\nContinuous integration # Since my source code was stored in GitHub and I could write my document using GitHub codespaces, I thought it would be nice to store the generated PDFs there as well. This last step is probably a bit overkill for most people, but I found it to be useful for managing different versions of my CV when applying for jobs.\nI built a workflow with GitHub actions making use of the handy LaTeX GitHub action, which runs the following jobs on all branches:\nPre-commit checks Builds the document to check that it still compiles Finally, once I’m happy with a specific version, I add a Git tag to trigger a new release with the PDF as an artifact.\nname: Build LaTeX document on: push: branches: - \u0026#34;*\u0026#34; tags: [\u0026#34;v*.*.*\u0026#34;] jobs: pre-commit: runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 # Install Perl for latexindent - uses: shogo82148/actions-setup-perl@v1 with: install-modules: YAML::Tiny File::HomeDir - uses: actions/setup-python@v3 - uses: pre-commit/action@v3.0.0 build: needs: pre-commit runs-on: ubuntu-latest steps: - name: Checkout repo uses: actions/checkout@v4 - name: Compile LaTeX document uses: xu-cheng/latex-action@v3 with: root_file: document.tex - name: Upload PDF artifact uses: actions/upload-artifact@v4 with: name: PDF path: document.pdf deploy: needs: build runs-on: ubuntu-latest if: github.ref_type == \u0026#39;tag\u0026#39; permissions: contents: write steps: - name: Download PDF artifact uses: actions/download-artifact@v4 with: name: PDF - name: Create release with PDF uses: softprops/action-gh-release@v1 with: files: document.pdf fail_on_unmatched_files: true ","date":"12 August 2024","externalUrl":null,"permalink":"/posts/latex/","section":"Posts","summary":"","title":"CI for your CV","type":"posts"},{"content":"I\u0026rsquo;m an engineer based in London, with experience from F1 and agtech, as well as a PhD in engineering. I\u0026rsquo;m highly skilled in building optimisation and machine learning models in Python. I have a passion for innovation and a strong attention to detail. I enjoy coming up with elegant solutions to challenging real-world problems.\nWhen I\u0026rsquo;m not coding, I enjoy running and cycling. I also like to try new craft beers and speciality coffees.\n","date":"18 April 2024","externalUrl":null,"permalink":"/about/","section":"About","summary":"","title":"About","type":"about"},{"content":"I personally love finding ways to reduce friction in my life easier and optimise my processes 1. The best tools are the ones which fit in easily to your workflow, and make you wonder how you worked without them before. Dev containers are one of those tools for me, and I’m going to explain why I like them so much.\nWhat are dev containers? # Dev containers are special Docker containers where you develop your code. This means that you just need Docker installed on your host machine, and then all dependencies are installed within the container.\nDev containers were popularised by VS code, but there is in fact an open specification. Dev containers can be used with multiple editors (incl. full-fat Visual Studio and IntelliJ based IDEs) and various cloud services (eg. GitHub Codespaces).\nYou define the environment through a Dockerfile (and possibly a docker-compose.yaml) and a configuration devcontainer.json file. These files configure the container for your project including installing any dependencies, and can be stored in your Git repo along with your source code.\nWhy do I need a dev container? # Zero-setup # The best-case scenario is when someone else has done the hard-work and defined the dev container for you. There’s no need to read “Get started” instructions for a new project, you just:\nPull the repo Launch VS code Build the container 2 Start developing This is particularly beneficial when your project has complex system requirements. One person works out the fiddly bits of how to set things up, and then shares it with everyone; either you pull the updated image, or rebuilding their container.\nThis proved invaluable at some of my previous company once when we started to build bits in different languages. I was able to get developing with a whole new eco-system of tools right away, without having to waste time configuring things.\nYou might wonder why this matters if you don’t work on many new projects and the system dependencies are stable. However, even in this scenario, dev containers can be helpful. For example, what happens if your machine breaks and you want to get started using a new one as quickly as possible?\nReproducibility # Dev containers help to fix a lot of “Works on my machine” problems. It can happen that the system dependencies to have changed since you were last working on your branch, and then you find your code no longer works. Without a dev container, you may end up spending a lot of time debugging before you eventually realise that there was some mismatch in the system dependencies between your main and development branches.\nOn the other hand, when you use dev containers, you could just rebuild the old version of the container to work on the old branch to carry on developing. And if there is a conflict in system dependencies, this should become apparent when you merge in your main branch.\nIsolation # The other major benefit of dev containers is that your dependencies are completely isolated from your host machine. You can\u0026rsquo;t accidentally break your machine - if you make a mistake in the dev container definition, you can just modify your Dockerfile and rebuild the container. It\u0026rsquo;s also trivial to completely remove all dependencies for a given project.\nYour projects are also completely isolated from each other, so that you never run into issues with conflicting system dependencies. I personally used to find that when developing locally, I would waste time trying to debug an issue before I realised I accidentally activated the wrong Python virtual environment. This is very unlikely to happen when using a dev container because it would (likely) only contain a single virtual environment.\nConfiguring a dev container # It can be quite overwhelming to configure a dev container for the first time. If you’re working on a project with a small team and not many system dependencies, then the benefits of a dev container will be limited. This means that it’s harder to justify the upfront investment in time.\nHowever, there\u0026rsquo;s no need to start from scratch. You can:\nMake use of pre-made dev container templates which you can then build on top of Add pre-made “features” to your definition, which carry out common tasks for you such as installing the AWS CLI In my experience, it becomes a lot easier to set up a dev container the second time around, because you can often copy a lot of the dev container definition from a previous project.\nUsing a dev container in the cloud # A nice bonus of containerising your development environment is that you can develop on any host machine, including ones in the cloud. The most obvious example is GitHub Codespaces, where you can spin up a new environment from a branch right from the GitHub UI. There are other cloud development providers such as Gitpod, Coder and CodeSandbox. I use Codespaces for personal projects quite a lot, since my machine is quite old and sometimes struggles, but you could even use this approach to develop on an iPad.\nA tool I have started to use recently is DevPod. This is an open-source tool which you install locally on your machine, and can then be used to spin up cloud dev environments almost as seamlessly as GitHub Codespaces. The key difference is that you can use any cloud provider such as AWS or GCP. This can work out to be much cheaper if your usage is high3, and allows you to further customise the host machine (eg adding GPUs).\nOr possibly even over-optimise? 😅\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nEven better, just pull the pre-built image\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nYou can read an interesting price comparison between AWS and Codespaces here\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"1 March 2024","externalUrl":null,"permalink":"/posts/devcontainers/","section":"Posts","summary":"","title":"Dev containers are awesome","type":"posts"},{"content":"I build things when I spot a problem worth fixing or want to understand how something works. Where it might be useful to others, I share it openly.\n","date":"1 March 2024","externalUrl":null,"permalink":"/projects/","section":"Projects","summary":"","title":"Projects","type":"projects"},{"content":"","date":"8 February 2024","externalUrl":null,"permalink":"/tags/computer-vision/","section":"Tags","summary":"","title":"Computer Vision","type":"tags"},{"content":"","date":"8 February 2024","externalUrl":null,"permalink":"/tags/deep-learning/","section":"Tags","summary":"","title":"Deep Learning","type":"tags"},{"content":"","date":"8 February 2024","externalUrl":null,"permalink":"/tags/deployment/","section":"Tags","summary":"","title":"Deployment","type":"tags"},{"content":" The aim of this project was for me to get some experience with computer vision problems. I found that the results were quite fun, so I decided to tidy up the code and deploy it. You can access the deployed Streamlit dashboard here.\nalxhslm/googly-eyes Automatically add Googly eyes to photos using ML Python 0 0 Problem description # The objective was to create a system which could automatically overlay googly eyes on top of people\u0026rsquo;s actual eyes in a given photo as shown below. I wanted it to work even if there was more than one person in the photo.\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?\u003e Googlify Simplifications # To simplify the problem, I made the following assumptions:\nWe only wish to detect human faces (and not pets) The faces are approximately aligned with the vertical axis of the photo, so the face detection step does need to be robust to misalignment The people will be facing the camera in the photos (or close to it), so we do not need to consider 3D geometric effects when placing the eyes Project plan # This problem can be decomposed into the following steps:\nBuild a build to identify eyes in photos of people Generate Googly eyes with random shapes and sizes Create a server which processes HTTP requests Create a UI to interact with the service I will now explain how I achieved each of these steps.\nEye identification # This is an object localisation problem since:\nWe need to identify the locations of eyes in the image We only need to classify a single class of objects The most obvious approach for identifying eyes is to use a CNN, trained on pre-labelled images of people’s faces.\nTraining a custom model # The first option I considered was to implement my own model. I could think of two possible architectures for localising the eyes:\nTrain a model to identify eyes directly from the photo Use a 2-stage identifier as follows: Stage 1: Identify faces Stage 2: Identify eyes from each face The latter approach has the potential to be more robust, since the eye identification model only has to consider faces, so is less likely to be \u0026ldquo;tricked\u0026rdquo; by other features in an image. However, it would be slower since two models need to be evaluated.\nIn either approach, we can make use of pre-trained image classification models for the main trunk of the model since:\nThey are often trained in datasets with people in so already have some capability to recognise faces and facial features The input layers of the network identify “features” which will transfer well to different applications The dataset we use to train network is very important. We must ensure that it is representative of the sorts of images which the users will upload. Therefore typical datasets for biometric identification would not be suitable. Instead we should use a dataset with photos in more natural environments. The LFW dataset seems most suitable.\nPre-trained models # Since the task of recognising faces and facial features is a very common one, we can make use of pre-existing models to perform this step for us. After a brief search, I came across many open-source models. The two most performant models are listed below.\nModel Paper Source code MTCNN PDF PyTorch / TensorFlow RetinaFace PDF PyTorch / TensorFlow RetinaFace generally performs slightly better, likely because it was trained with an augmented set of labels including 1k 3D vertices.\nI chose to make use of the existing RetinaFace model, since this model would likely be much more accurate than any model I could train, without spending a lot of time on data processing and model training. This also allowed me to focus my time on the other aspects of the system.\nI converted the Tensorflow model to use the Tensorflow-lite runtime instead. This required updating the model and pre-processing steps to use fixed image dimensions. This has two main advantages:\nIt significantly reduces the size of the docker images It reduces execution time Googly-eye generation # Once we have detected the position of the eyes, we need to add the googly eyes in the correct location. I used the ImageDraw module to perform the image manipulation and draw the eyes. However, I still needed to decide how to size the eyes and where to place the pupils.\nEye size # To ensure that the googly eyes are of an appropriate size, independent of image dimensions or the distance from the face to the camera, I computed the eye-to-eye separation distance for a given face from:\n$$ \\Delta_{eye2eye}=\\sqrt{(x_r-x_l)^2+(y_r-y_l)^2} $$\nwhere \\(x_r, y_r\\) are the pixel co-ordinates of each eye. I then set the radius of the googly eyes \\(r_e\\) by:\n$$ r_e =\\gamma\\frac{\\Delta_{eye2eye}}{2} $$\nwhere \\(\\gamma\\) is a scaling parameter in the range \\(0\u0026lt;\\gamma\u0026lt;1.0\\). I set the default value to 0.5.\nPupil size # The pupil size \\(r_p\\) was set based on the eye size from:\n$$ r_p=\\lambda r_{e} $$\nwhere \\(\\lambda\\) is a random variable sampled from the following distribution:\n$$ \\lambda \\sim U(\\lambda_1, \\lambda_2) $$\nwhere the default values for the parameters \\(\\lambda_1\\) and \\(\\lambda_2\\) were set to 0.4 and 0.6 respectively.\nPupil position and orientation # To randomise the position of the pupil, the position was set by the following equation:\n$$ x_p = x_e+ (r_e-r_p) \\sin \\theta $$\n$$ y_p = y_e+ (r_e-r_p) \\cos \\theta $$\nwhere \\(\\theta\\) is the random orientation sampled from the following distribution:\n$$ \\theta \\sim U(0,2\\pi) $$\nLocal development # The first stage was get the system running locally on my machine. This was split into two components:\nServer to handle HTTP requests and perform the image manipulation Dashboard acting as a user-friendly interface for making requests to the server Server # In order to handle the HTTP requests, I created a server using Flask. I created a single end-point specified as follows:\nBody Response Image Edited image Googly eye parameters Locations of detected faces Both the body and response of the post request are in JSON format. In both cases, the photo is serialized as a Base64 string. Additional parameters for the eye and pupil size can be included in the request body. This allows the user to override the value of any of these settings to personal preference. The locations of the faces are returned for debugging purposes.\nThe server runs through the following steps:\nDeserialize the request body to extract the image and parameters Call the RetinaFace model to identify the faces in the images Overlay googly eyes on each of the faces Serialize the edited image and combine with the identified faces to form the response body All image processing and manipulation was then performed in memory, so no images are ever stored to disk on the server.\nWhen running the server in Docker, I used the Waitress web server. The Python dependencies are managed using Poetry and the server is encapsulated within a Docker container.\nDashboard # To allow the user to interact with the server more easily, I build a minimal dashboard using Streamlit which allows the user to:\nUpload a photo Adjust parameters for the googly eyes Download the modified photo The dashboard then performs the following steps:\nGets the settings from the information entered by the user Makes the HTTP request to the server to add the googly eyes Overlay the identified faces for debugging purposes (if requested) Displays the result and adds a download link The dashboard is hosted in a separate Docker container, with its own smaller set of dependencies using Poetry. The network connection to the server is configured using Docker compose.\nCloud deployment # Now that I had got everything working locally, the next stage was to deploy it to the cloud.\nServer # Since the server was already contained within a Docker container and contained only a single end-point, it was quite simple to convert it into an AWS Lambda function. I used an AWS function URL to expose the AWS Lambda, with IAM authentication. This had the advantage that I did not need to manage any compute resources.\nThe main challenge I ran into was the the AWS Python Lambda Docker images were based on a version of Amazon Linux with out-of-date system dependencies for the version of Tensorflow I was using. To get around this, I built a custom image using the process described here, based on a Debian-based Python image from Docker Hub.\nI found that the AWS Lambda required 3GB of RAM to run the RetinaFace model reliably. However, it is quite slow due to the server being quite underpowered, and therefore needs a large 60s timeout.\nDashboard # Unfortunately the Streamlit dashboard I built already in the dashboard subdirectory could not be deployed to Streamlit cloud as-is. This is because the dashboard used uses files from the shared common directory but Streamlit cloud dashboards only have access to files in the same directory.\nTo get around this limitation, I created a wrapper module app.py at the root of the repo, which in turn calls the existing dashboard code. This ensures that the deployed dashoard has access to the entire repo. This setup supports both local development and cloud deployment, without any code duplication.\nThe architecture of the production system is shown below.\ngraph TD subgraph Streamlit Cloud subgraph Wrapper module A end end E[User]-.-\u003eA A[Dashboard]--Image--\u003eB[Lambda Function] B--Edited image--\u003eA B--Image--\u003eD[RetinaFace] D---\u003eF F[Googlifier]--Edited image--\u003eB subgraph AWS subgraph Docker container B D F end end ","date":"8 February 2024","externalUrl":null,"permalink":"/projects/googly-eyes/","section":"Projects","summary":"","title":"Googly Eyes","type":"projects"},{"content":"When building ML applications, managing the various datasets \u0026amp; models can be quite a headache, which is where MLOps tools come in. The current landscape is still quite immature, and there are endless tools with different capabilities, aiming to streamline different aspects of the machine learning lifecycle. This makes it quite difficult to assess which tools are suitable for your use case.\nIn this blog post, I will share my experiences using ClearML, which is one of the most popular MLOps tools. I will discuss the features which we found to be useful at my current company, as well as some of the limitations we ran into. I hope this will help guide others when choosing an MLOps tool for their team.\nWhat is ClearML? # ClearML is a popular end-to-end MLOps tool that covers the whole ML pipeline within a single open-source platform. It offers a suite of tools to manage datasets, experiments, and models, and also track them all in a unified web interface. There are some other tools in this space, and I have put a brief comparison of the other popular ones below. Compared to the other options, ClearML is a more heavyweight tool, although not quite as popular.\nClearML MLflow Weights and Biases Open-source :white_check_mark: :white_check_mark: :white_check_mark: Self-host :white_check_mark: :white_check_mark: :white_check_mark: Cloud-hosted :white_check_mark: :x: :white_check_mark: Dataset management :white_check_mark: :x: :x: Experiment tracking :white_check_mark: :white_check_mark: :white_check_mark: Model registry :white_check_mark: :white_check_mark: :x: Model deployment :white_check_mark: :white_check_mark: :x: Why did we choose ClearML? # Flexible experiment tracking # One of the major strengths of ClearML is that it is quite generic, making it suitable for a wide range of ML applications. ClearML allows users to record custom metadata, log live metrics and store plots and other rich data in the results.\nFor example, at my company, we used ClearML for the classical use-case of training models implemented in JAX, where we tracked hyperparameters, and recorded the trained parameters and other metrics. However, we also needed to run computationally intensive simulations with these models. In this case, we used ClearML to record settings for the simulations, and track the progress so we could monitor the results in real-time. ClearML was able to support both use-cases very well.\nRun experiments anywhere # ClearML experiments can run on any machine where you can run the agent. We initially used ClearML to track experiments run locally on our machines. Once the number of models we needed to maintain increased, we then built the capability to run experiments in the cloud.\nThe value of tracking local experiments is often underestimated during the model development process, but it allowed us to collaborate more easily and isolate performance regressions. Being able to train models in the cloud then allowed us to iterate more quickly, but tracking experiments locally was still a big step forward 1.\nVibrant community # ClearML offers comprehensive and well-structured documentation, particularly for the most-used APIs. It was easy for us to find the information we needed to get started and make the most out of the platform.\nAdditionally, ClearML has an active community on Slack, providing a platform for users to seek support, share ideas, and collaborate with other users. Whenever we ran into issues or found bugs, we found that the ClearML team and wider community was quick to respond.\nBatteries included # Unlike some other tools such as Weights and Biases, ClearML goes beyond just experiment tracking, and includes features for dataset management, orchestration and even deployment. This allows you to have a centralised platform for the whole ML pipeline, without having to integrate additional tools such as using DVC to version datasets or KServe for deployment, which could be seen as introducing additional complexity.\nWhether this is seen as positive or negative will likely depend on your use case. However, if you need the full set of capabilities offered by ClearML and have a small team, it is much quicker to get up and running with a single tool.\nAffordable pricing for small teams # One of the main reasons we chose ClearML was that it offers a Community Edition which allows users to self-host on their own infrastructure. This allows us to keep all of the data in-house and reduce the running costs significantly, which is obviously a significant advantage for a start-up.\nClearML also offers a hosted solution if you don’t want to manage your own infrastructure. This is priced based on the number of users, which means it is affordable for small teams, although this might become expensive once your team scales.\nWhat limitations have we found with ClearML? # Some key features are paywalled # The free tier and self-hosted options are generally quite generous. However, not all the features you might expect are included, and the documentation on which features are included could be made clearer. For example, one limitation we ran into was that autoscalers are not available in the free tier, so we had to build extra tooling in-house to distribute experiments between workers.\nPerhaps more disappointingly, other important features such as SSO or Kubernetes integration are not available even in the pro tier, and require one of the enterprise tiers2. Therefore it is important to check if the specific features required for your use case are included in the relevant tier you intend to use.\nDocumentation is patchy # While ClearML’s documentation is generally very clear and quite comprehensive, there are certain more advanced or esoteric areas of the API which are less well-documented. For example, the standard ClearML API does not include support for managing queues (for orchestrating tasks over workers), and instead recommends using the generic HTTP API client. However, the documentation for this API is quite limited so some trial and error was required for us to get this to work.\nThe ClearML documentation also sometimes fails to explain certain concepts beyond the key ones such as Tasks and Models particularly well. For example, we wanted to run groups of simulations, where each simulation was for a different scenario. The solution turned out to be to use function tasks, which allow a parent task to spawn multiple sub-tasks, but it was not clear from the documentation alone that this would achieve what we needed.\nFortunately it seems that ClearML has since improved the documentation in this and other areas, so perhaps this will become less of an issue going forward.\nVisualisation options are limited # ClearML allows you to view metrics from a single experiment very quickly and easily from the UI. You can also generate custom plots and store them in the results for the experiment, which can then also be viewed from the UI.\nWhilst you can also very easily compare metrics from different experiments on the same chart, you are unable to view custom plots from different experiments on the same chart 3. This means that your visualisation options for comparing experiments is limited to simple scalar metrics.\nWe overcame this limitation by building custom dashboards using Streamlit to fetch data from the ClearML database, and then visualise results using our own custom plots. This worked quite well for us, but this obviously requires additional maintenance. It would be cleaner if you could analyse experiments in more detail inside the ClearML UI itself.\nConclusions # With its extensive documentation and active community support, ClearML is an affordable and flexible solution for smaller teams building out ML pipelines. Although ClearML has some shortcomings and certain restrictions in the free/self-hosted tier, it has a much greater set of capabilities compared to the alternatives, and continues to evolve and improve over time. Overall, ClearML has proven to be a valuable MLOps tool at my company, and is an option worth considering for your team.\nIt is worth noting that this also applies to other MLOps tools in general, and not just ClearML.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nThere are both scale and enterprise tiers which both have custom pricing.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nUnfortunately it doesn\u0026rsquo;t seem like this is something which will be supported anytime soon going by this long-standing GitHub issue.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"30 December 2023","externalUrl":null,"permalink":"/posts/clearml/","section":"Posts","summary":"","title":"ClearML: A review","type":"posts"},{"content":"","date":"19 December 2023","externalUrl":null,"permalink":"/tags/website/","section":"Tags","summary":"","title":"Website","type":"tags"},{"content":"I built this website as a platform for giving updates on my projects and sharing my thoughts. As someone who builds software and interacts with GitHub on a daily basis, the natural choice for hosting a website was GitHub Pages. It’s completely free and you don’t even have to create another account. The only limitation is that it can only host static sites, but that’s not really an issue for simple text-heavy websites such as blogs.\nWhat are static site generators? # Even after making this decision, there are many different potential tools for generating the site. Unless you’re a web developer, the best option for most people is to use a static-site generator (SSG). These tools allow you to write your content in markdown, and then automatically generate the website without having to touch HTML. I found that there are broadly two types of SSGs:\n\u0026ldquo;Simple\u0026rdquo; generators \u0026ldquo;Advanced\u0026rdquo; generators Jekyll (written in Ruby) Gatsby Hugo (written in Go) Astro 11ty (written in JavaScript) The \u0026ldquo;advanced\u0026rdquo; options also support some dynamic content and are targeted for “proper” websites. I therefore instantly discounted these options, as they would introduce unnecessary complexity for a blog in my opinion.\nOf the \u0026ldquo;simple\u0026rdquo; options, Jekyll is anecdotally the most popular, likely because this has been the default option from the start of GitHub Pages. Hugo is a newer option and has recently overtaken Jekyll in terms of stars. 11ty is newer still but much less popular.\nI decided to immediately discount 11ty since it appears to be more targeted towards those with more frontend experience and has a smaller community, so my remaining choices were Jekyll and Hugo. It seems that the consensus is that Jekyll is “simpler” and appears to still be considered to be the “default” option. However, things have changed a lot since GitHub Pages was first introduced and my experience with Hugo has been positive (so far). Therefore, I thought it would be useful to share my reasoning for choosing Hugo in case it helps someone else.\nSo why did I choose Hugo? # Hugo ships as a single executable # Jekyll is shipped as a Ruby package, which means you first need to install Ruby on your machine. I remember how much of a pain it was to set up a Python environment for the first time1, so the idea of having to install Ruby just seemed like an unnecessary headache.\nHugo on the other hand ships as a single executable so you can do brew install hugo. It’s just so much easier, and was one of the biggest reasons why Hugo initially appealed to me.\nHugo’s own documentation is excellent # Since Jekyll is over 10 years old, and you can find loads of examples online to help resolve issues if you get stuck. With Hugo, when I do search for issues that arise, I often get results for Jekyll mixed in, so this is certainly one area where Hugo is weaker.\nFortunately, Hugo’s website itself is very good. The API documentation is quite extensive, and the forums are active. I will concede though that the documentation is quite technical and assumes a lot of prior knowledge, so there are a lot of concepts to get your head around initially (although I don’t know if this is any worse than Jekyll).\nWhile I think it is slightly harder to find help at the moment in Hugo, I think this issue is generally overstated. Given the popularity of Hugo, this is something which should continue to improve over time.\nHugo shortcodes are deceptively powerful # I read in multiple places that the leaning curve for custom templates is harder with Hugo than Jekyll. For simple templating, Jekyll’s includes are arguably simpler and easier to read using Liquid tags.\nHugo’s shortcodes are more verbose than Jekyll includes, but they support more complex logic. I’m sure you can achieve even more with a Jekyll plug-in if you are proficient in Ruby, but I was able to create a few custom components with shortcodes pretty quickly without any prior experience with Go.\nTo me it seems that shortcodes strike a sensible middle ground which is powerful enough for most users.\nDeploying Hugo websites with Github Actions is easy # If you use Jekyll, you can publish websites to GitHub Pages using the Ruby environment provided by GitHub Pages Ruby gem. This means that you just push your changes to a specific branch and the website automatically gets built and deployed without any additional configuration.\nThere is an alternative method using GitHub actions, where you configure a workflow to run when certain criteria are met (normally a commit to the main branch). The action then builds your website and deploys it2.\nWith Hugo on the other hand, you have to use GitHub actions. If you’ve never used CI before, this might seem daunting, but Hugo has clear instructions on how to configure everything. They also provide a template workflow, which you can simply copy over to your repo. It took me about 10 minutes to first deploy the first version of this website.\nPersonally I prefer using GitHub actions because:\nIt is more explicit how the site is deployed, and you have full control over when and how it is built and deployed It avoids having to learn how the Ruby environment provided by GitHub Pages works It allows me to also configure build checks on every commit (even those not on main)3 Hugo is faster # This is unlikely to really matter for most people, but since Hugo is written in Go which is a compiled language, it builds very quickly and faster than Jekyll. It’s nice to know you are very unlikely to run into issues in the future.\nSee obligatory reference to this XKCD comic\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nIt seems that GitHub actions will likely become the new default for Jekyll too, given that GitHub still hasn’t updated pages to use Jekyll 4.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nYou can view my modified workflow here.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"19 December 2023","externalUrl":null,"permalink":"/posts/why_hugo/","section":"Posts","summary":"","title":"Why Hugo instead of Jekyll?","type":"posts"},{"content":" This was another personal ML project, in which I wanted to gain some experience working with unstructured data. With all of the recent advancements in generative AI, also thought it would be interesting to use some form of generative model. The source code is available on GitHub.\nalxhslm/hand-writing-generation A GenAI app to generate hand-written characters Python 1 0 Objective # The aim of this project is to:\nTrain a model to classify hand-written alphanumeric characters Generate synthetic hand-written characters using the same model I used the standard MNIST dataset which contains 70000 images of the numeric characters (0-9), and supplemented it with this Kaggle dataset which contains 370000 images of the alphabetical characters (A-Z).\nModel # I decided to use a Variational autoencoder since dimensionality reduction is an area I find personally interesting. This is also one of the few widely used models uniting deep learning and Bayesian methods.\nTypically a VAE is used for unsupervised problems, as a non-linear method of dimensionality reduction. However, we have class labels in this case, so I instead implemented a semi-supervised VAE. As we will see later, this will allow us to choose which character we reconstruct with the encoder.\nThe model was implemented using pytorch. The encoder, decoder and loss function will be discussed below.\nEncoder # This part of the model predicts the latent variables corresponding to a given image. Rather than generating a point estimate for the latent variables, the encoder predicts the posterior distribution \\(p(z|x)\\):\n$$ p(z|x) \\approx N(\\mu=q_{x,1}(x), \\sigma^2 = e^{q_{x,2}(x))}$$\nwhere \\(q_{x,1}(x)\\) and \\(q_{x,2}(x)\\) are approximated by a neural network within the encoder.\nThe encoder also predicts the class probabilities \\(p(y|x)\\):\n$$p(y|x) \\approx q_y(x)$$\nwhere \\(q_y(x)\\) is also approximated as a neural network within the encoder.\nThe architecture of the encoder is shown in the diagram below:\nIt has the following layers:\nConvolutional layer + ReLU (input size 28x28x1, output size 14x14x32) Convolutional layer + ReLU (input size 14x14x32, output size 7x7x32) Flatten (input size 7x7x32, output size 1x(49x32)) To generate class labels \\(y\\): Linear + ReLU (input size 49x32, output size 128) Linear + Softmax (input size 128, output size 36) 1 To predict mean and variance of latent variables \\(z\\): Linear (input size 1x(49x32), output size 1x(64x2)) The dimension of the latent variables \\(z\\) was set to 64. Decoder # This part of the model reconstructs an image from class labels and latent variables. We first sample from the posterior distribution:\n$$ z \\sim N(\\mu=q_{x,1}(x), \\sigma^2 = e^{q_{x,2}(x)})$$\nand we have already computed the class probabilities \\(p(y|x)\\). The final step is to then reconstruct an image from this information. The decoder has the following architecture, which is very similar to the encoder but in reverse:\nThis has the following layers:\nFrom class labels \\(y\\) logits: Linear + ReLU (input size 36, output size 128) Linear (input size 128, output size 49x32) From latent variables \\(z\\): Linear (input size 64, output size 49x32) Unflatten (input size 49x32, output size 7x7x32) Deconvolutional layer + ReLU (input size 7x7x32, output size 14x14x32) Deconvolutional layer + Sigmoid (input size 14x14x32, output size 28x28x1) 2 Loss function # The last key component is the loss function. This is what allows us to train the model, and ensure that the encoder can accurately predict the posterior distribution and the decoder can accurately reconstruct images.\nFor a supervised VAE like this, we need 3 loss terms:\nReconstruction loss to enforce that the decoder can accurately reconstruct characters from the latent variables. This was implemented using BCEWithLogitsLoss. Categorical loss to enforce that the model can accurately classify characters by predicting class probabilities \\(p(y|x)\\). This was implemented using CrossEntropyLoss. Kullback-Leibler divergence loss to enforce that the encoder accurately predicts the posterior on the latent variables \\(p(z|x)\\). 1 \u0026amp; 3 would be needed for an unsupervised VAE as well, but 2 is an additional loss needed for the semi-supervised nature of this problem. The KL-loss was implemented as follows:\ndef kl_div_loss_fun(z_mean: torch.Tensor, log_z_var: torch.Tensor) -\u0026gt; torch.Tensor: return -0.5 * torch.sum(1 + log_z_var - z_mean.pow(2) - log_z_var.exp()) / z_mean.shape[0] where z_mean is given by \\(q_{x,1}(x)\\) and log_z_var is \\(q_{x,2}(x)\\).\nInteractive dashboard # In order to get a better feel for how the model works, I created a Streamlit dashboard which allows you to interact with the model. You can access the dashboard here and have a go yourself.\nThis dashboard goes through the following steps:\nYou begin by drawing a few characters which gives us some observations \\(x_{t}\\) This allows the encoder to produce an estimate of the posterior distribution \\(p(z|x_{t})\\), which effectively allows the model to “learn” your writing style. You can then generate arbitrary hand-written characters by sampling the distribution \\(p(x) = p(x|z, y)p(z)\\) where \\(p(z)=p(z|x_{t})\\) and the value of \\(y\\) determines which character is generated. By varying how you draw, you can change the style of characters which the model produces. Here are some examples of the sort of hand-written characters the model can synthesise:\nPrompt Description Generated images Narrow Wide Light Bold Italic Note that in practice no Softmax is applied here since this is implicitly included as part of the CrossEntropyLoss, which leads to improved numerical stability\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nNote that in practice, no Sigmoid was applied here since this is implicitly included as part of the BCEWithLogitsLoss, which again leads to improved numerical stability\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"1 December 2023","externalUrl":null,"permalink":"/projects/hand-writing-generation/","section":"Projects","summary":"","title":"Hand Writing Generation","type":"projects"},{"content":"","date":"1 December 2023","externalUrl":null,"permalink":"/tags/modelling/","section":"Tags","summary":"","title":"Modelling","type":"tags"},{"content":"","date":"21 November 2023","externalUrl":null,"permalink":"/series/coffee-rating-prediction/","section":"Series","summary":"","title":"Coffee Rating Prediction","type":"series"},{"content":" Background # In a previous post in this series, we trained a model which was able to predict how highly rated a coffee would be on CoffeeReview.com based on the following features:\nOrigin Roaster and roasting style Price Flavour profile I will use this dataset from Kaggle which contains ratings for ~1900 coffees.\nModel deployment # To allow us to interact with the model, it was deployed to the cloud within a server as shown below. Separating the model like this allows it to be updated independently from other systems which use it. The individual components will be discussed further below.\ngraph TD E[User]-.-\u003eA A[Streamlit Web App]--Request--\u003eB[AWS Lambda] B--Rating--\u003eA B--Features--\u003eD[Model] D--Prediction--\u003eB subgraph Docker container D end Docker # The chosen model was wrapped within a Docker container which contains all dependencies including Python, as well as the required packages using poetry. The image was built locally and then uploaded to Amazon Container Registry. To update the model, we just need to build a new version of the Docker container.\nAWS Lambda # An AWS Lambda function was then configured to use this Docker container and call the model. This is a serverless product, which means that:\nWe don\u0026rsquo;t have to maintain any infrastructure We are only billed when the model is actually used To allow us to interact with the model, an HTTP endpoint was created using a Lambda URL. In order to secure the endpoint and only allow requests from the dashboard, it was authenticated using IAM. When a HTTP request containing the features is received, the data is passed to the model which generates predictions, and the rating is returned in the response.\nInteractive dashboard # In order to be able to interact the model and predict ratings for arbitrary coffees, I created a Streamlit dashboard which you can access here. The user enters the meta data about the coffee, and the dashboard then generates a dict of the required features.\nThe dashboard then makes an HTTP request to the server with this dict in the message body, and receives the rating in the response. The predicted rating is then displayed in the dashboard. Have a play and see how your favourite coffee fares!\n","date":"21 November 2023","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/deployment/","section":"Projects","summary":"","title":"Deployment","type":"projects"},{"content":"","date":"21 November 2023","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":" Background # In a previous post in this series, we performed experimental data analysis to understand the distribution of the rating and get some insight as to which features might be important.\nThe next step is to use this information to develop and train a model on the data which is capable of predicting the ratings.\nFeature selection # We have quite a few features available, but not all will lead to a significant improvement of the accuracy of the model. We will therefore begin by assessing the importance of the different features in a quantitative way, so that we include the minimum number needed to generate accurate predictions.\nCorrelation # We can first compute the correlation coefficient between the numerical features and the rating. In this case, the only numerical feature is the \u0026quot;price_per_100g\u0026quot;:\ndf[[\u0026#34;price_per_100g\u0026#34;]].corrwith(df[\u0026#34;rating\u0026#34;]) Feature Correlation coefficient price_per_100g 0.241615 This is a moderately high value, suggesting that the price does significantly influence the rating and should therefore be included as an input to the model.\nMutual information # For the categorical features, we can instead analyse the mutual information:\nfrom sklearn.metrics import mutual_info_score mutual_info = pd.Series({k: mutual_info_score(df[k], df[\u0026#34;rating\u0026#34;]) for k in [\u0026#34;roaster\u0026#34;, \u0026#34;roast\u0026#34;, \u0026#34;roaster_country\u0026#34;, \u0026#34;country_of_origin\u0026#34;, \u0026#34;region_of_origin\u0026#34;]}) mutual_info.sort_values(ascending=False) Feature Mutual information roaster 0.670450 country_of_origin 0.159215 roaster_country 0.068317 roast 0.046098 region_of_origin 0.033712 We can see that the roaster has by far the biggest influence, which was expected based on the results from the EDA. This is followed by the country of origin. Interestingly, the region of origin has a much lower mutual information score than the country of origin, suggesting that we cannot use the region in place of the country of origin.\nWe can also compute the same metric for the different flavours:\nmutual_info_flavours = pd.Series({k: mutual_info_score(df[k], df[\u0026#34;rating\u0026#34;]) for k in FLAVOURS}) mutual_info_flavours.sort_values(ascending=False) Feature Mutual information resinous 0.046254 fruity 0.036157 spicy 0.020958 nutty 0.012289 acidic 0.008529 floral 0.006981 chocolate 0.006747 herby 0.006218 carbony 0.005304 caramelly 0.004110 We can see that the most important flavours are the \u0026ldquo;resinous\u0026rdquo; and \u0026ldquo;fruity\u0026rdquo; flavours, and have a similar level of significance to the \u0026quot;roast\u0026quot; feature, which again agrees with the results from the EDA. The \u0026ldquo;spicy\u0026rdquo; and \u0026ldquo;nutty\u0026rdquo; together provide about the same information as the \u0026ldquo;fruity\u0026rdquo; flavour. The rest of the flavours are much less significant.\nSummary # Based on this brief analysis, we will choose the following features:\nPrice per 100g Roaster Roaster country Roast Country of origin Flavours: \u0026ldquo;resinous\u0026rdquo;, \u0026ldquo;fruity\u0026rdquo;, \u0026ldquo;spicy\u0026rdquo;, \u0026ldquo;nutty\u0026rdquo; Feature engineering # Now that we have selected the features to use, we need to transform them into a form which the model can accept.\nPrice # The price of the coffee was found to have a very long tail. Many models make assumptions about the data, including that the features are normally distributed. Therefore we may get better performance if we transform this feature to be closer to a normal distribution. The log transformation is one such transformation which can achieve this:\ndf[\u0026#34;price_per_100g\u0026#34;].apply(np.log1p).hist() Roaster # Since the previous analysis showed that there is evidence that the roasters significantly influences the rating, the model needs a feature giving it this information. We cannot simply convert the roaster using one-hot encoding as there are too many different values. Let us instead only include the most common roasters (those with \u0026gt; 10 coffees).\nroasters = df[\u0026#34;roaster\u0026#34;].value_counts() popular_roasters = sorted(roasters[roasters \u0026gt; 10].index) Let\u0026rsquo;s save this information to roasters.json for later use.\nroaster_map = df[[\u0026#34;roaster\u0026#34;, \u0026#34;roaster_country\u0026#34;]].groupby(\u0026#34;roaster\u0026#34;).first()[\u0026#34;roaster_country\u0026#34;] roaster_info = {\u0026#34;known_roasters\u0026#34;: roaster_map.to_dict(), \u0026#34;popular_roasters\u0026#34;: popular_roasters} with open(\u0026#34;data/roasters.json\u0026#34;, \u0026#34;w\u0026#34;) as f: json.dump(roaster_info, f, indent=4) We can now use this information to engineer roaster features with a smaller number of unique values.\ndf[\u0026#34;popular_roaster\u0026#34;] = df[\u0026#34;roaster\u0026#34;].where(df[\u0026#34;roaster\u0026#34;].apply(lambda r: r in popular_roasters), \u0026#34;Other\u0026#34;) If we find these features have a strong influence on the model, we need to be careful when applying the model to new coffees from unknown roasters. Even if the coffee is from a well-known roaster, they will have a \u0026quot;roaster\u0026quot; value of \u0026ldquo;Other\u0026rdquo; if they are not present in the training set. Flavours # Let\u0026rsquo;s now combine the flavours into a single column.\ndf[\u0026#34;flavours\u0026#34;] = df.apply(lambda coffee: [flavour for flavour in FLAVOURS if coffee[flavour]], axis=1) One-hot encoding # The remaining task is to encode the categorical variables. We will use a one-hot encoding scheme, which can be easily implemented using the DictVectorizer:\nfrom sklearn.feature_extraction import DictVectorizer dv = DictVectorizer(sparse=False) dv.fit(X_train.to_dict(orient=\u0026#34;records\u0026#34;)) Note that this can natively handle the \u0026quot;flavours\u0026quot; column which contains a list of flavours. Every time we train or evaluate a model we will need to apply this transformation.\nSummary # We can now combine these steps to assemble the input feature matrix X and target vector y.\nFEATURES = [\u0026#34;price_per_100g\u0026#34;, \u0026#34;popular_roaster\u0026#34;, \u0026#34;roaster_country\u0026#34;, \u0026#34;roast\u0026#34;, \u0026#34;country_of_origin\u0026#34;] FLAVOURS = [\u0026#34;fruity\u0026#34;, \u0026#34;resinous\u0026#34;, \u0026#34;spicy\u0026#34;, \u0026#34;nutty\u0026#34;] X = df[FEATURES].copy() X[\u0026#34;price_per_100g\u0026#34;] = X[\u0026#34;price_per_100g\u0026#34;].apply(np.log1p) X[\u0026#34;flavours\u0026#34;] = df.apply(lambda coffee: [flavour for flavour in FLAVOURS if coffee[flavour]], axis=1) X = dv.transform(X.to_dict(orient=\u0026#34;records\u0026#34;)) y = df[\u0026#34;rating\u0026#34;] Building a model # Validation framework # We must first split the dataset into train/validation/test sets, where I have chosen a 60%/20%/20% distribution. I have set the random_state parameter to 1 to guarantee reproducibility.\nfrom sklearn.model_selection import train_test_split X_train, X_test = train_test_split(X, test_size=0.2, random_state=1) y_train, y_test = train_test_split(y, test_size=0.2, random_state=1) We will train the model using the train sets, and finally evaluate using the test set. We will use K-folds validation to guard against overfitting to the validation set when performing hyperparameter selection.\nfrom sklearn.model_selection import KFold kf = KFold(n_splits=5, shuffle=True, random_state=1) Linear regression # Let\u0026rsquo;s start with the simplest model which is a linear regressor. I will use the Ridge model which included L2 regularisation to prevent overfitting. I will train the model for multiple values of the regularisation weight parameter alpha, and record the losses on the training and validation sets.\nfrom sklearn.linear_model import Ridge from sklearn.metrics import mean_squared_error def train_ridge_using_kfold(model: Ridge, X: pd.DataFrame, y: pd.Series) -\u0026gt; tuple[float, float]: mse_train = [] mse_val = [] for _, (train_index, val_index) in enumerate(kf.split(X)): X_train = X.iloc[train_index, :] y_train = y.iloc[train_index] X_val = X.iloc[val_index, :] y_val = y.iloc[val_index] model.fit(_transform(X_train), y_train) mse_train.append(mean_squared_error(y_train, model.predict(_transform(X_train)))) mse_val.append(mean_squared_error(y_val, model.predict(_transform(X_val)))) return np.mean(mse_train), np.mean(mse_val) scores_linear = pd.DataFrame(columns=[\u0026#34;train\u0026#34;, \u0026#34;validation\u0026#34;]) for alpha in [0.01, 0.03, 0.1, 0.3, 1.0, 3.0, 10.0]: model = Ridge(alpha=alpha) mse_train, mse_val = train_ridge_using_kfold(model, X_train, y_train) scores.loc[alpha, :] = pd.Series({\u0026#34;train\u0026#34;: mse_train, \u0026#34;validation\u0026#34;: mse_val}) fig = scores_linear.plot(log_x=True, labels={\u0026#34;index\u0026#34;:\u0026#34;alpha\u0026#34;, \u0026#34;value\u0026#34;:\u0026#34;loss\u0026#34;}) We see that if we use too high a value of alpha, the RMSE starts to increase because the regularisation term is too strong and forces too simple a model. It seems that a suitable value of alpha is 10.0, since this gives a relatively low loss on both the validation and test sets.\nWe can now fit a model on the combined train and validation set.\nlinear_model = Ridge(alpha=10.0) linear_model.fit(X_train, y_train) If we plot the predicted distribution of ratings from this model, we see that it captures the central part of the distribution quite well and there is no bias. However, the model fails to predict the more extreme ratings, and therefore the peak around the median is slightly higher.\npd.DataFrame( {\u0026#34;true\u0026#34;: y_train, \u0026#34;prediction\u0026#34;: np.round(linear_model.predict(X_train), decimals=0)} ).hist() We can get a bit more insight by evaluating the importance of the different features using permutation_importance.\nfrom sklearn.inspection import permutation_importance r = permutation_importance(linear_model, X_train, y_train, n_repeats=10, random_state=0) linear_importances = pd.Series(dict(zip(dv.get_feature_names_out(), r.importances_mean))) linear_importances[linear_importances.abs().sort_values(ascending=False).index] Feature Importance price_per_100g 0.177400 popular_roaster=Other 0.078086 flavours=resinous 0.070506 country_of_origin=Kenya 0.065933 country_of_origin=Ethiopia 0.044702 roaster_country=Taiwan 0.037096 flavours=fruity 0.033723 popular_roaster=Kakalove Cafe 0.024013 popular_roaster=Hula Daddy Kona Coffee 0.018847 country_of_origin=Panama 0.017545 We can see that the biggest influence is the price, which was expected given the high correlation we observed. Certain countries styles have a large importance, which is expected given that the mutual information from this feature was quite high. The flavours have a significant but lower importance compared to the other features.\nGradient-boosted trees # Another type of model which performs well on tasks like this are random forests. Here I will use XGBoost which also performs gradient boosting to further improve performance. We will train a model for an increasing number of estimators (ie decision trees), and for increasing maximum tree depth (set by the max_depth parameter).\nimport xgboost as xgb def train_xgb_using_k_fold(model: xgb.XGBRegressor, X: pd.DataFrame, y: pd.Series) -\u0026gt; pd.DataFrame: mse = [] for _, (train_index, val_index) in enumerate(kf.split(X)): X_train = X.iloc[train_index, :] y_train = y.iloc[train_index] X_val = X.iloc[val_index, :] y_val = y.iloc[val_index] eval_sets = { \u0026#34;train\u0026#34;: (_transform(X_train), y_train), \u0026#34;validation\u0026#34;: (_transform(X_val), y_val), } model.fit(_transform(X_train), y_train, eval_set=list(eval_sets.values())) results = model.evals_result() mse.append(pd.DataFrame({k: results[f\u0026#34;validation_{i}\u0026#34;][\u0026#34;rmse\u0026#34;] for i, k in enumerate(eval_sets)})) return sum(mse) / len(mse) scores_depth = {} for max_depth in [1, 2, 3, 4, 5]: xgb_params = { \u0026#39;max_depth\u0026#39;: max_depth, \u0026#39;min_child_weight\u0026#39;: 1, \u0026#39;objective\u0026#39;: \u0026#39;reg:squarederror\u0026#39;, \u0026#39;seed\u0026#39;: 1, \u0026#39;verbosity\u0026#39;: 1, } model = xgb.XGBRegressor(**xgb_params, eval_metric=\u0026#34;rmse\u0026#34;) scores[max_depth] =train_xgb_using_k_fold(model, X_train, y_train) import plotly.graph_objects as go fig = go.Figure() for i, (depth, df) in enumerate(scores_max_depth.items()): fig.add_trace(go.Scatter(x = df.index, y=df[\u0026#34;train\u0026#34;], name=f\u0026#34;{depth} (train)\u0026#34;, line_dash=\u0026#34;dash\u0026#34;, line_color=COLORS[i])) fig.add_trace(go.Scatter(x = df.index, y=df[\u0026#34;validation\u0026#34;], name=f\u0026#34;{depth} (val)\u0026#34;, line_color=COLORS[i])) fig.update_layout(xaxis_title=\u0026#34;n_estimators\u0026#34;, yaxis_title=\u0026#34;rmse\u0026#34;, legend_title_text = \u0026#34;max_depth\u0026#34;) fig.show() We can see that the model is able to achieve a lower RMSE for greater values for max_depth. However, this does not come with a corresponding decrease in validation error, indicating that there is overfitting. A suitable value would be max_depth=1 or 2 since these have the lowest difference between the training and validation loss. Using a max_depth of 2 does lead to a lower validation loss, so this is preferable, so long as the number of estimators is limited. A suitable selection would max_depth=2 with 10 estimators.\nWe can now retrain a model with these parameters, but for multiple values of the eta parameter which controls the \u0026ldquo;learning rate\u0026rdquo; (ie how strongly each new estimator aims to compensate for the previous ones):\nscores_eta = {} for eta in [0.01, 0.03, 0.1, 0.3, 1.0]: xgb_params = { \u0026#39;max_depth\u0026#39;: 2, \u0026#39;n_estimators\u0026#39;: 10, \u0026#34;eta\u0026#34;: eta, \u0026#39;min_child_weight\u0026#39;: 1, \u0026#39;objective\u0026#39;: \u0026#39;reg:squarederror\u0026#39;, \u0026#39;seed\u0026#39;: 1, \u0026#39;verbosity\u0026#39;: 1, } model = xgb.XGBRegressor(**xgb_params, eval_metric=\u0026#34;rmse\u0026#34;) scores_eta[eta] = train_xgb_using_k_fold(X_train, y_train) fig = go.Figure() for i, (eta, df) in enumerate(scores_eta.items()): fig.add_trace(go.Scatter(x = df.index, y=df[\u0026#34;train\u0026#34;], name=f\u0026#34;{eta} (train)\u0026#34;, line_dash=\u0026#34;dash\u0026#34;, line_color=COLORS[i])) fig.add_trace(go.Scatter(x = df.index, y=df[\u0026#34;validation\u0026#34;], name=f\u0026#34;{eta} (val)\u0026#34;, line_color=COLORS[i])) fig.update_layout(xaxis_title=\u0026#34;n_estimators\u0026#34;, yaxis_title=\u0026#34;rmse\u0026#34;, legend_title_text = \u0026#34;eta\u0026#34;) We can see that if we set too low a value, the model is not able to achieve as low a loss. However, if it is too high, the model overfits and the training loss continues to decrease without reducing the validation loss. It appears that we should select eta = 0.3 to give the best compromise.\nWe can now train a model with these final parameters on the combined training and validation sets:\nxgb_params = { \u0026#39;max_depth\u0026#39;: 2, \u0026#39;n_estimators\u0026#39;: 10, \u0026#34;eta\u0026#34;: 0.3, \u0026#39;min_child_weight\u0026#39;: 1, \u0026#39;objective\u0026#39;: \u0026#39;reg:squarederror\u0026#39;, \u0026#39;seed\u0026#39;: 1, \u0026#39;verbosity\u0026#39;: 1, } xgb_model = xgb.XGBRegressor(**xgb_params, eval_metric=\u0026#34;rmse\u0026#34;) xgb_model.fit(X_train, y_train, eval_set=[(X_train, y_train)]) results = xgb_model.evals_result() If we plot the histogram of the model predictions, we see that this model also fails to capture the very low or high ratings in the same way as the linear model.\npd.DataFrame( { \u0026#34;true\u0026#34;: y_train, \u0026#34;prediction\u0026#34;: np.round(xgb_model.predict(X_train), decimals=0), } ).hist() As with the linear models, can get a bit more insight by evaluating the importance of the difference features.\nr = permutation_importance(xgb_model, X_train, y_train, n_repeats=10, random_state=0) xgb_importances = pd.Series(dict(zip(dv.get_feature_names_out(), r.importances_mean))) xgb_importances[xgb_importances.abs().sort_values(ascending=False).index] feature importance price_per_100g 0.172254 flavours=resinous 0.090398 popular_roaster=Other 0.069107 flavours=fruity 0.038945 popular_roaster=El Gran Cafe 0.031304 roaster_country=Taiwan 0.024728 country_of_origin=Kenya 0.023421 country_of_origin=Ethiopia 0.020484 popular_roaster=Kakalove Cafe 0.013997 country_of_origin=Panama 0.010082 We see that the price continues to have the biggest influence. We also see that the \u0026ldquo;resinous\u0026rdquo; and \u0026ldquo;fruity\u0026rdquo; flavours continue to be significant here, and are actually more important than certain countries of origin. The roaster feature also play a significant role, but interestingly the roaster which features here is different from the linear model.\nModel selection # Finally can evaluate the losses of both models on the test dataset, and see which model is most accurate.\nmodels = {\u0026#34;linear\u0026#34;: linear_model, \u0026#34;xgb\u0026#34;: xgb_model} scores_comparison = pd.DataFrame(dtype=float) for name, model in models.items(): loss_train = mean_squared_error(y_train, model.predict(X_train), squared=False) loss_test = mean_squared_error(y_test, model.predict(X_test), squared=False) scores[name] = pd.Series({\u0026#34;train\u0026#34;: loss_train, \u0026#34;test\u0026#34;: loss_test}) scores_comparison.transpose().plot.bar() Both models achieve similar losses on the training set, but there is difference in performance on the test set. We can see that the XGBoost model has a much larger difference between test losses. This suggests that this model has overfit to the training set.\nWe can see that the two models both predict a similar distribution of ratings with shorter tails than the ground truth distribution.\npd.DataFrame( {\u0026#34;true\u0026#34;: y_test} | {name: np.round(model.predict(X_test), decimals=0) for name, model in models.items()} ).hist() This suggests that the models are failing to predict the highest/lowest scores due to some systematic error such as:\nLack of information in the features (eg perhaps we need more detailed information about the origin) Inconsistencies in the review process (eg different reviewers with different preferences) Investing this is beyond the scope of this project, since the linear model achieves the objective with sufficiently good performance.\nConclusions # We have shown that it is possible to predict how highly rated a coffee would be on CoffeeReview.com based purely on information about the coffee and a linear model. We have shown that the biggest influencers on rating are:\nPrice Flavour: if a coffee is fruity or resinous Origin: If a coffee is from East Africa If a coffee is from certain roasters We showed that the models could predict the rating to quite a high degree of accuracy, but struggled to predict particularly low or highly rated coffees. However, it produces an unbiased estimate.\nOverall, the two models have very similar performance on the test set. The linear regression model is preferred since it is simpler and has slightly better performance.\nIn the next post in this series, we will actually deploy the trained model to the cloud.\n","date":"13 November 2023","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/model/","section":"Projects","summary":"","title":"Modelling","type":"projects"},{"content":"This was a personal ML project, and the goal was to familiarise myself with XGBoost and AWS Lambda. I enjoy drinking nice coffee, so I chose a topic which I hoped would help me buy better coffee in the future. The source code is available on GitHub and you can access the hosted Streamlit dashboard here.\nalxhslm/coffee-rating-prediction A simple ML app to predict the rating of a given coffee Python 0 0 ","date":"9 November 2023","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/","section":"Projects","summary":"","title":"Coffee Rating Prediction","type":"projects"},{"content":" Objective # The objective of this project is to be able to predict how highly rated a coffee would be on CoffeeReview.com based purely on information about the coffee such as:\nOrigin Roaster and roasting style Price Flavour profile I will use this dataset from Kaggle which contains ratings for ~1900 coffees.\nData processing # I will begin by loading in the data into a pd.DataFrame and also renaming the columns to more clearly distinguish between the country of origin and roaster country.\nimport pandas as pd df = pd.read_csv(\u0026#34;./data/simplified_coffee.csv\u0026#34;) for col in [\u0026#34;name\u0026#34;, \u0026#34;roaster\u0026#34;, \u0026#34;roast\u0026#34;, \u0026#34;loc_country\u0026#34;, \u0026#34;origin\u0026#34;, \u0026#34;review\u0026#34;]: df[col] = df[col].astype(\u0026#34;string\u0026#34;) df[\u0026#34;review_date\u0026#34;] = pd.to_datetime(df[\u0026#34;review_date\u0026#34;]) df = df.rename(columns={\u0026#34;loc_country\u0026#34;: \u0026#34;roaster_country\u0026#34;, \u0026#34;100g_$\u0026#34;: \u0026#34;price_per_100g\u0026#34;, \u0026#34;origin\u0026#34;: \u0026#34;country_of_origin\u0026#34;}) df.head() name roaster roast roaster_country country_of_origin price_per_100g rating review_date review Ethiopia Shakiso Mormora Revel Coffee Medium-Light United States Ethiopia 4.70 92 2017-11-01 Crisply sweet, cocoa-toned. Lemon blossom, roa\u0026hellip; Ethiopia Suke Quto Roast House Medium-Light United States Ethiopia 4.19 92 2017-11-01 Delicate, sweetly spice-toned. Pink peppercorn\u0026hellip; Ethiopia Gedeb Halo Beriti Big Creek Coffee Roasters Medium United States Ethiopia 4.85 94 2017-11-01 Deeply sweet, subtly pungent. Honey, pear, tan\u0026hellip; Ethiopia Kayon Mountain Red Rooster Coffee Roaster Light United States Ethiopia 5.14 93 2017-11-01 Delicate, richly and sweetly tart. Dried hibis\u0026hellip; Ethiopia Gelgelu Natural Organic Willoughby\u0026rsquo;s Coffee \u0026amp; Tea Medium-Light United States Ethiopia 3.97 93 2017-11-01 High-toned, floral. Dried apricot, magnolia, a\u0026hellip; We should check for any NaNs:\ndf.isna().sum() name 0 roaster 0 roast 12 roaster_country 0 country_of_origin 0 price_per_100g 0 rating 0 review_date 0 review 0 The roast column is the only one containing NaNs. Since there are only 12 missing values, we could just remove these rows. However, since most coffees have the same roasting style (as will see later), let us fill with the modal value.\ndf[\u0026#34;roast\u0026#34;] = df[\u0026#34;roast\u0026#34;].fillna(df[\u0026#34;roast\u0026#34;].mode().iloc[0]) Let\u0026rsquo;s fix a typo in the roaster country for one coffee.\ndf[\u0026#34;roaster_country\u0026#34;] = df[\u0026#34;roaster_country\u0026#34;].str.replace(\u0026#34;New Taiwan\u0026#34;, \u0026#34;Taiwan\u0026#34;) Some roasters such as \u0026ldquo;El Gran Cafe\u0026rdquo; are duplicated since they entered with slightly different spellings in different rows (eg sometimes with the accented é and sometimes with a plain e). I will rename the roasters consistently by replacing these characters:\nreplace = {\u0026#34;’s\u0026#34;: \u0026#34;\u0026#39;s\u0026#34;, \u0026#34;é\u0026#34;: \u0026#34;e\u0026#34;, \u0026#34;’\u0026#34;: \u0026#34;\u0026#39;\u0026#34;} for k, v in replace.items(): df[\u0026#34;roaster\u0026#34;] = df[\u0026#34;roaster\u0026#34;].str.replace(k, v) We should also verify that the information about each roaster (in this case only country) is consistent across all coffees from the same roaster.\ndef _assert_identical_values(df: pd.DataFrame) -\u0026gt; pd.Series: assert (df.iloc[1:, :] == df.iloc[0, :]).all().all() return df.iloc[0, :] Region of origin # Most coffees come from certain regions of the world, and the coffees from each region tend to be similar in flavour profile (eg African coffees are typically more acidic and South American coffees more nutty.) Therefore it may be useful to categorise the countries by region.\nThe mapping from regions to countries has been manually compiled and stored in regions.json. We need to load this JSON file, invert the mapping so that it goes from country to region, and then engineer a new column for the region.\nwith open(\u0026#34;./data/regions.json\u0026#34;, \u0026#34;r\u0026#34;) as f: REGIONS = json.load(f) regions = {} for r, countries in REGIONS.items(): for c in countries: regions[c] = r df[\u0026#34;region_of_origin\u0026#34;] = df[\u0026#34;country_of_origin\u0026#34;].map(regions).fillna(\u0026#34;Other\u0026#34;) Flavour notes # As it stands, we cannot glean any information from the review column as it is unstructured. Let\u0026rsquo;s begin by analysing the keywords present in the reviews using the WorldCloud package:\nfrom wordcloud import WordCloud, STOPWORDS COFFEE_WORDS = {\u0026#34;cup\u0026#34;, \u0026#34;notes\u0026#34;, \u0026#34;finish\u0026#34;, \u0026#34;aroma\u0026#34;, \u0026#34;hint\u0026#34;, \u0026#34;undertones\u0026#34;, \u0026#34;mouthfeel\u0026#34;, \u0026#34;structure\u0026#34;, \u0026#34;toned\u0026#34;} word_cloud = WordCloud(collocations=False, width = 1000, height = 500, background_color=\u0026#39;white\u0026#39;, stopwords=set(STOPWORDS) | COFFEE_WORDS).generate( \u0026#34; \u0026#34;.join(df[\u0026#34;review\u0026#34;]) ) plt.imshow(word_cloud) Note that we have had to remove common coffee-related nouns, which is assumed provide no information about the particular coffee. We can see that the most common words relate to the flavour of the coffee such as \u0026ldquo;acidity\u0026rdquo; or \u0026ldquo;chocolate\u0026rdquo;. This suggests that we can extract some features for the different flavours in the coffee.\nUsing this insight and the coffee flavour wheel, we can manually define some flavours and corresponding keywords which are stored in flavours.json.\nwith open(\u0026#34;./data/flavours.json\u0026#34;, \u0026#34;r\u0026#34;) as f: FLAVOURS = json.load(f) We can now engineer boolean features for each flavour.\ndef rating_contains_words(review: str, keywords: list[str]) -\u0026gt; bool: words = extract_words(review) for w in keywords: if w in words: return True return False for flavour, keywords in FLAVOURS.items(): df[flavour] = df[\u0026#34;review\u0026#34;].apply(rating_contains_words, args=(keywords,)) It is also interesting to check how many flavours the different coffees have. If we have done a good job at defining the flavour keywords, we would expect that mode coffees would:\nHave at least some flavours since this is a key component of any review Not have an excessive number of flavours, as this would indicate we have chosen too \u0026ldquo;common\u0026rdquo; keywords df[list(FLAVOURS.keys())].sum(axis=1).hist() Indeed, this appears to be the case. All coffees have at least 2 flavours, and in fact most coffees have ~6 flavours.\nDistributions of each feature # We will begin by exploring the distribution of each feature in the dataset.\nRating # We can see that the ratings appear to be approximately normally distributed. However, the median rating is surprisingly high at ~94% and the standard deviation is low, so that the effect rating range ranges roughly from 85-100.\ndf[\u0026#34;rating\u0026#34;].hist() Price # The distribution for the price of the coffee is shown below.\ndf[\u0026#34;price_per_100g\u0026#34;].hist() There is a very long tail, due to a few very expensive coffees. This suggests that there may be a benefit in applying the log transformation when we come to fit the model.\nRoasting style # The vast majority of the coffees have the medium-light roast type. This large bias in the dataset may make it challenging for a model to detect any impact of roasting style on coffee rating.\ndf[\u0026#34;roast\u0026#34;].hist() Roaster country # If we look at the value counts, we see that most of the coffees are from US roasters.\ndf[\u0026#34;roaster_country\u0026#34;].value_counts() roaster_country United States 774 Taiwan 339 Hawai\u0026rsquo;i 77 Guatemala 24 Hong Kong 9 Japan 8 England 7 Canada 5 Australia 1 China 1 Kenya 1 If we look at the distribution of pricing for the most common countries, we see that the distribution is quite different in each country. In particular, the distribution of coffees from US roasters has a much more pronounced peak at the lower price level. This likely indicates that there is some bias in the dataset. Given that CoffeeReview is based in the US, they have reviewed a disproportionate number of more affordable coffees from US roasters.\ncountries = [\u0026#34;United States\u0026#34;, \u0026#34;Taiwan\u0026#34;, \u0026#34;Guatemala\u0026#34;] df[df[\u0026#34;roaster_country\u0026#34;].apply(lambda c: c in countries)].histogram(\u0026#34;price_per_100g\u0026#34;,by=\u0026#34;roaster_country\u0026#34;) This strong bias towards coffees roasted in the US means that it is unclear how well any model trained on this data will generalise to coffees roasted outside the US. In addition, the number of different countries present is very small, and we cannot for example, predict if a coffee from a German roaster would be more or less likely to be highly rated since there are no coffees from German roasters in the dataset. Origin # Country of origin # We will first examine the different countries of origin in the dataset:\ndf[\u0026#34;region_of_origin\u0026#34;].hist() As expected, most of the coffees come from the largest coffee producing countries in the world, almost a third are from Ethiopia alone.\nRegion of origin # We also plot the regions of origin:\ndf[\u0026#34;region_of_origin\u0026#34;].hist() Almost all examples are from one of the following regions which are the major coffee producing regions of the world:\nEast Africa such as Ethiopia or Kenya Central or South America such as Colombia or Guatemala There are also a lot of coffees from Hawaii, which is likely again because the data is from a US-based website.\nFlavour # In order to examine the popularity of the different flavours, we can plot the histogram:\ndf[list(FLAVOURS.keys())].sum().divide(df.shape[0]).sort_values(ascending=False).plot.bar() We can see that the most common flavours are:\nCaramelly Acidic Fruity Chocolate Intuitively, this makes sense as these are the sorts of flavours seen mentioned on packets of coffee.\nInfluence of features on rating # We can now move onto trying to identify any influence of the different features on the rating of the coffee.\nPrice # To visualise the influence of price on the rating, we can simply produce a scatter plot:\ndf.plot.scatter(x=\u0026#34;price_per_100g\u0026#34;, y=\u0026#34;rating\u0026#34;) There appears to be some positive correlation between the two variables, although there is a fair amount of scatter so it is clear that there are other mechanisms at play here as well. This suggests that either:\nPrice is genuinely an indicator of quality Price biases the reviewers There is evidence of diminishing returns from increasing the price of the coffee, as the relationship between price and rating starts to flatten off.\nRoaster # If we look at the highest and lowest rated coffees, we see that they are dominated by certain roasters.\ndf.loc[df[\u0026#34;rating\u0026#34;] \u0026gt; 96, [\u0026#34;name\u0026#34;, \u0026#34;roaster\u0026#34;]].groupby(\u0026#34;roaster\u0026#34;).count() roaster count Barrington Coffee Roasting 1 Bird Rock Coffee Roasters 1 Dragonfly Coffee Roasters 1 Hula Daddy Kona Coffee 1 JBC Coffee Roasters 3 Kakalove Cafe 1 Paradise Roasters 2 df.loc[df[\u0026#34;rating\u0026#34;] \u0026lt; 90, [\u0026#34;name\u0026#34;, \u0026#34;roaster\u0026#34;]].groupby(\u0026#34;roaster\u0026#34;).count() roaster count El Gran Cafe 8 Other 4 To gain further insight, we can plot the average rating for the coffees with the most coffees:\nroasters = df[\u0026#34;roaster\u0026#34;].value_counts() popular_roasters = sorted(roasters[roasters \u0026gt; 10].index) df.groupby(\u0026#34;roaster\u0026#34;)[\u0026#34;rating\u0026#34;].mean()[popular_roasters].plot.bar() This is shown below, with the average rating across all coffees shown in black.\nWe can see that there is significant variation in the average rating for each roaster. For example, \u0026ldquo;El Gran Cafe\u0026rdquo; has a particularly low average rating around 3.0 below the average rating. This suggests that either:\nCertain roasters find the best/worst coffees or roast them particularly well The reviewers favour/dislike certainer roasters Roasting style # To assess if roasting style has an impact on the rating, we can plot the histogram grouping by the roasting style:\ndf.hist(\u0026#34;rating\u0026#34;, by=\u0026#34;roast\u0026#34;) It is clear that dark and medium-dark roasted coffees are often rated more poorly, since the tail has a significant skew to the left for these roasting styles. We can also see that the lighter roasted coffees have a much higher median rating, with medium roasted coffees somewhere in between.\nWe can plot the mean rating for each roasting style in the same way as for the roasters:\ndf.groupby(\u0026#34;roast\u0026#34;)[\u0026#34;rating\u0026#34;].mean().plot.bar() As expected, the darker roasting styles have a much lower rating on average, which is because some of the darker roasted coffees are related particularly poorly. However, there is a much smaller difference between the light and medium-light roasting styles.\nOrigin # Since there are many countries, we will analyse the influence of region of origin instead of country. We can plot the histogram of rating grouping by each region:\ndf.hist(\u0026#34;rating\u0026#34;, by=\u0026#34;region_of_origin\u0026#34;) We can see that the distribution for Central American coffees has a left-leaning tail, and the median is slightly lower than the other regions. However, we can glean more information by looking at the mean rating for each region of origin:\ndf.groupby(\u0026#34;region_of_origin\u0026#34;)[\u0026#34;rating\u0026#34;].mean().plot.bar() This highlights that the East African coffees are more highly rated on average, and Central American are less highly rated. Therefore, there is evidence that the origin may have some influence on the rating of a coffee. However, the effect does not appear to be as strong as the roaster for example, since the difference between the highest and lowest average ratings is only around 0.5 compared to around 3.0 for the roaster.\nFlavour # To investigate the impact of flavour on rating, we can compute the average rating for coffees with and without each flavour, and compute the difference. If a flavour has a big impact on rating, we would expect to see a large difference.\nrating_with_flavour = pd.Series({f: df.loc[df[f], \u0026#34;rating\u0026#34;].mean() for f in FLAVOURS}) rating_without_flavour = pd.Series({f: df.loc[~df[f], \u0026#34;rating\u0026#34;].mean() for f in FLAVOURS}) (rating_with_flavour-rating_without_flavour).hist(\u0026#34;region_of_origin\u0026#34;, by=\u0026#34;roast\u0026#34;) We can see that for most flavours, the difference is quite small. However, it appears that fruitiness has a large positive influence and resinous a large negative influence on the rating.\nConclusions # In this article, we has processed the coffee dataset and performed analysis to establish that:\nThe dataset has a strong bias towards: Coffees from US roasters Coffees from East Africa or South America Medium-light roasted coffees We have detected which factors may be more likely to impact the rating: There is some definite correlation between price and rating Certain roasters have much higher and lower average ratings for their coffees, suggesting that this does impact the rating The roast style, origin and flavour profile all appear to have some impact on the rating, but it does not appear that the relationship is as strong as the roaster In the next post, we will use the insight gained from this analysis to engineer features, and then training a predictive model.\n","date":"9 November 2023","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/eda/","section":"Projects","summary":"","title":"Exploratory data analysis","type":"projects"},{"content":"A few years ago, I started a daily journal habit. I found that preferred to do this digitally because my hand-writing is terrible, and it is easier to look back when compared to a paper notebook. Initially I used the Journey app because:\nIt was the only cross-platform option at the time (via the web app) It was a lot cheaper than the alternatives (since there was a one-off payment option) Once I switched to using primarily Apple devices, I decided to take a look again at the different options. Although I liked Journey, it did not have the same polish as Day One, so I made the switch. I had around 2 years of journal entries to migrate, but whilst Day One offers Journey import on Android1, they don\u0026rsquo;t on MacOS or iOS. This meant that, rather frustratingly and unnecessarily, my data was stuck in Journey\nTherefore I decided to create my own tool to perform the migrate automatically. Fortunately Day One and Journey both store their entries in JSON format under the hood. Once I reverse engineered the schema (with some trial and error), the conversion logic was relatively simple. The code is open source and available at the repo below.\nalxhslm/journey2dayone Python script to convert Journey diary entries to Day One format Python 12 4 So the developer clearly already knows how to import the data.\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"3 January 2021","externalUrl":null,"permalink":"/projects/dayone/","section":"Projects","summary":"","title":"Day One Importer","type":"projects"},{"content":"","date":"3 January 2021","externalUrl":null,"permalink":"/tags/journal/","section":"Tags","summary":"","title":"Journal","type":"tags"},{"content":" alxhslm/HarmLAB Implementation of the harmonic balance method in MATLAB MATLAB 20 6 The Harmonic Balance method is an advanced numerical technique for analysing the oscillatory response of nonlinear dynamics systems. During my PhD I wanted to apply this method to efficiently compute the response of rotor-bearing systems, but there weren\u0026rsquo;t any freely available libraries. I therefore wrote my own, making this technique available to all. The implementation is completely generic so can be applied to many domains, ranging from life sciences to engineering problems.\n","date":"5 May 2019","externalUrl":null,"permalink":"/projects/harmlab/","section":"Projects","summary":"","title":"HarmLAB","type":"projects"},{"content":"","date":"5 May 2019","externalUrl":null,"permalink":"/tags/simulation/","section":"Tags","summary":"","title":"Simulation","type":"tags"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/comparison_hist/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/comparison_losses/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/flavour_hist/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/linear_hist/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/linear_losses/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/mean_rating_by_flavour/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/mean_rating_by_origin/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/mean_rating_by_roast/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/mean_rating_by_roaster/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/num_flavours_hist/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/origin_country_hist/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/origin_region_hist/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/price_hist/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/price_log_hist/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/rating_against_price/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/rating_hist/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/rating_hist_by_origin/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/rating_hist_by_roast/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/roast_hist/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/roaster_country_hist/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/trees_hist/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/trees_losses_depth/","section":"Projects","summary":"","title":"","type":"projects"},{"content":" ","externalUrl":null,"permalink":"/projects/coffee-rating-prediction/charts/trees_losses_eta/","section":"Projects","summary":"","title":"","type":"projects"},{"content":"","externalUrl":null,"permalink":"/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"},{"content":"","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"Please use the form below if you would like to contact me about my work.\nYour name Your email Your message Send ","externalUrl":null,"permalink":"/about/contact/","section":"About","summary":"","title":"Contact","type":"about"},{"content":" Experience Education Carbon Re Sept 2024 to Present Senior Machine Learning Engineer Decarbonising cement production using AI Website\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?\u003e H2GO Power Apr 2024 to Aug 2024 Senior Research Scientist Building AI optimisation algorithms to better manage hydrogen assets Website\nOptimal Labs Sept 2020 to Jan 2024 Senior Engineer Deploying fully autonomous greenhouses using state-of-the-art control algorithms Website\n\u003c?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?\u003e McLaren Racing Aug 2012 to Nov 2016 Simulation Engineer Developing the driver-in-the-loop simulator and offline simulation tools for the F1 team Website\nImperial College London Nov 2016 to Sept 2020 PhD Engineering Investigated non-linear dynamic phenomena in rotating machinery using advanced modelling and numerical simulation techniques. Sponsored by Rolls Royce.\nThesis: Non-linear vibration transmission through rolling-element bearings in rotating machines\nDownload University of Cambridge Oct 2008 to Jun 2012 MEng Engineering Specialising in dynamics \u0026 control with modules covering modelling, simulation, numerical methods and optimisation. Master's thesis in collaboration with Renault F1. Thesis: Stability and controllability of an F1 race car\nDownload Publications # Journal papers # Haslam, A., Schwingshackl, C. W., Rix, A. I. J. (2020). \u0026ldquo;A parametric study of an unbalanced Jeffcott rotor supported by a rolling-element bearing\u0026rdquo;. In: Nonlinear Dynamics 99, pp. 2571–2604. [DOI] Braghieri, G., Haslam, A., Sideris, M., Timings, J., Cole, D. (2017). \u0026ldquo;Quantification of road vehicle handling quality using a compensatory steering controller\u0026rdquo;. In: Journal of dynamic systems, measurement, and control 139.3, p. 031010. [DOI] Conference proceedings # Haslam, A., Schwingshackl, C. W., Rix, A. (2020). \u0026ldquo;Experimental investigation of non-linear stiffness behaviour of a rolling-element bearing\u0026rdquo;. In: 12th International Conference on Vibrations in Rotating Machinery. CRC Press, pp. 411–422. [Link] Schwingshackl, C. W., Muscutt, L., Szydlowski, M., Haslam, A., et al. (2020). \u0026ldquo;Asynchronous rotor excitation system (ARES) – A new rotor dynamic test facility at Imperial College London\u0026rdquo;. In: 12th International Conference on Vibrations in Rotating Machinery. CRC Press, pp. 400-410. [Link] Haslam, A., Schwingshackl, C. W., and Rix, A. I. J. (2019). \u0026ldquo;Analysis of the Dynamic Response of Coupled Coaxial Rotors\u0026rdquo;. In: Rotating Machinery, Vibro-Acoustics \u0026amp; Laser Vibrometry, Volume 7: Proceedings of the 36th IMAC, A Conference and Exposition on Structural Dynamics 2018. Springer International Publishing, pp. 53–65. [DOI] Haslam, A. et al. (2019). \u0026ldquo;Nonlinear System Identification for Joints Including Modal Interactions\u0026rdquo;. In: Non- linear Dynamics, Volume 1: Proceedings of the 36th IMAC, A Conference and Exposition on Structural Dynamics 2018. Springer International Publishing, pp. 79–99. [DOI] Skills \u0026amp; Technologies # Machine Learning NumPy Pandas Scikit-learn PyTorch JAX Optimisation \u003c?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?\u003e CVXPY Optuna CasADi Mathematical Modelling Numerical methods Physics-based modelling Data visualisation Matplotlib Plotly Streamlit Experiment Management ClearML MLflow Software Development Python Poetry Poetry Pytest Git GitHub Actions CircleCI Cloud \u0026amp; Infrastructure Docker GCP AWS ","externalUrl":null,"permalink":"/about/cv/","section":"About","summary":"","title":"Curriculum vitae","type":"about"}]