Designed and deployed multi-tenant infrastructure using AWS CDK, orchestrating provisioning across PostgreSQL and AWS via a State Machine. — 2 days → 5 minutes tenant configuration.
Built full test automation suite (unit, integration, E2E) integrated into CI/CD, with developer metrics surfaced via Jira. — +20% test coverage.
Developed a shared data validation library in Pydantic adopted by multiple engineering teams. — reduced cross-team data defects/inconsistencies.
Introduced
uvpackage manager. — 12 min → 4.5 min deployment time.Designed pre-fetch architecture to execute downstream Lambdas in the event of a user-click in the frontend.
Hello, I’mJack Harding
Backend & Cloud EngineerAWS, Python, GoRecent work in LLM tooling, RAG and data pipelinesVancouver
Vancouver, BCExperience
Where I’ve worked, and what I built there.
Designed and developed a real-time data dashboard built in streamlit visualising hormonal cycle data and in-app notes.
Kick-started company data strategy, defining goals and aligning stakeholders across product and engineering.
Worked in a medical-device regulatory environment where data correctness was a compliance requirement, not a preference.
Aided in the maintenance of TypeScript codebase.
Designed time-series prediction for cycle length using ARMA model in statsmodels and pandas.
Worked independently developing and maintaining SQL Server databases, including schema design and query performance tuning.
Integrated SAP master docs into Lakeland Dairies SQL Server tables using SSIS and .NET C# scripts.
Redesigned and developed company website in WordPress alongside managing director and senior management.
Introduced git source control to company for SQL & C# code artifacts.
Built a mobile game in Unity/C# as one of a four-person team, shipping to both Android and iOS.
Projects
Self-directed work.
Go MCP server exposing OpenBeta's public climbing GraphQL API as agent tools.
Replaced the original
crags_withinbounding-box tool with acragsNearpoint+radius proximity search returning leaf crags only. — ~180 areas at leaf zoom → leaf crags only; removed a class of upstream 504 timeouts.Round-trip cost instrumentation: a
CountTransportHTTP transport wrapper counting upstream requests against a counter carried in the *request context*, so fan-out is attributed to a single tool call rather than to the process.Token-cost harness measuring the token footprint of every tool result with
tiktoken, to catch payload regressions deterministically without running a model. — 888-line token-distribution dataset across the tool corpus.Migrated eval run tracking to MLflow — JSONL stays the record, MLflow is the viewer — replacing a bespoke Go VCS-stamp run-identity scheme that worked but was out-of-scope to maintain. — ~1900 lines of analysis-layer code deleted and replaced with one JSONL→MLflow exporter.
Markdown → two-column PDF CV generator (WeasyPrint + CSS, no LaTeX), with one directory per application pairing
cv.mdagainst the job posting it was written for, so tailoring is diffable in git rather than living in binary documents. — 20 application variants generated to date.Single-source-of-truth vault: every role, project and skill recorded once with honest capability levels (Production / Hands-on / Exposure / None), so a CV bullet cannot claim a level the vault does not back. — 122 records across 4 roles and 14 projects.
Publishing is a section allowlist, not a filter: only vault sections 0–4 are parsed at all, so private sections (framings, interview stories, known gaps, role preferences) cannot leak by omission.
Test suite asserts the privacy exclusions against the *real* vault rather than a fixture, because a section-allowlist regression is the one failure in the repo that cannot be walked back once published. — 122 tests.
gh-digest: extracts commit counts, diff sizes, merged PR bodies, languages and contribution windows from public repos into plain text, deliberately running no LLM during extraction so the record stays a data export rather than a draft. — 42 repos digested.
Built a GitOps deployment loop for a FastAPI service: an ArgoCD Application syncing Kubernetes manifests straight from the Git repo, with the ArgoCD UI reconciling against the repo as source of truth.
Containerised the app and added a CI job building and pushing the image to Docker Hub, scoping registry credentials as job-level secrets.
Moved manifests into a dedicated k8s directory, repointed the ArgoCD Application at it, and added YAML linting for both the Argo and Kubernetes manifests.
REST API with JWT auth, rate limiting (slowapi), async PostgreSQL via SQLAlchemy, structured JSON logging over a 100k-row route dataset.
Benchmarked three pagination strategies (offset, keyset, cursor) against an unpaginated baseline. — offset: −600ms mean, −2.2s p95 across 50 queries at 50km radius.
Geohash-based proximity search over 100k generated routes. — ~4x faster than brute-force haversine at 1k points; degrades past 20km due to unindexed prefix scans (documented).
Composite B-tree index on
(lat, lon)for bounding-box queries, benchmarked against an unindexed baseline; documented that p95 barely moved because large bbox queries are dominated by payload size, not row lookup. — ~halved median latency (−13ms at p50, 67% reduction).Explicit cache invalidation on write: POST invalidates the route's geohash namespace across all cached pages; documented the residual POST/GET race condition as unresolved rather than papering over it.
AWS CDK-deployed Lambda benchmark comparing 4 implementations of D8 flow-direction computation over DEM (digital elevation model) data — Go, Go-Tiled (goroutine-parallelised), Python loop, and Python vectorised (NumPy) — swept across memory tiers (512MB–3000MB), 15 cold + 15 warm iterations per function via CloudWatch REPORT log parsing.
Go-Tiled variant splits row ranges across
sync.WaitGroup/runtime.NumCPU()goroutine workers.Published findings in a Medium article with a concrete decision framework: Go for latency-sensitive/API-backed Lambdas.
The benchmark harness built for this project became the
lambda-benchpip package.
pip-installable CLI benchmarking AWS Lambda cold/warm start performance across Python and Go runtimes for geospatial workloads.
Full test suite (CLI, export, invoker, parser, plotting, stats, runner, integration) with CI publish pipeline; installable via
uv/pip; includes a Go-vs-Python worked example.
End-to-end serverless event ingestion pipeline: AWS CDK, Lambda, API Gateway, Kinesis Firehose, S3.
Full test pyramid in a uv-workspaces monorepo.
Athena/Glue query layer over processed data, feeding Metabase; partition projection means Athena discovers new date/hour partitions without crawler runs, so the crawler schedule was cut back since the schema is largely static. — Glue Crawler: ~$0.44/DPU-hour, ~$1.76/month at ~weekly (4 runs/month) cadence; Athena priced at $5/TB scanned.
Pydantic schema validation on ingested events plus API Gateway request validation. — 39 parameterized unit tests for event validation.
Full monthly cost estimate published for demo-scale usage: S3 storage + requests, Lambda (1M invocations across 3 functions), Kinesis Firehose (10GB ingested), Glue Crawler, Athena (10GB scanned), API Gateway. — ~$6.65/month total at demo scale.
Rule-based classifier predicting avalanche exposure ratings for alpine routes.
Processes raw DEM rasters and vector terrain data into structured, model-ready features using GDAL and GeoPandas.
Improved automated test coverage on the classifier package. — 76% test coverage.
Evaluated FlowPy vs D8 flow-routing algorithms for runout modelling. — FlowPy: ~13 min per area × 177 areas — not viable; D8 adopted instead.
Validated the ported AutoATES-v2 model via confusion matrix to confirm reproducibility against the original; rules-based classifier validated on three examples with mixed results.
RAG chatbot over Bellingcat's investigative journalism archive: web-scraped all Bellingcat articles (
newspaper3kfor parsing), embedded titles into a ChromaDB collection, queried via Streamlit chat UI.Optimised for slow build times by caching both the scraped-article file and the vector database.
Separate PEFT/LoRA fine-tuning exercise (same project) on FLAN-T5-base for dialogue summarisation, evaluated with ROUGE-L against a full fine-tune and the zero-shot base model.
Second Bellingcat tool: Streamlit dashboard showing Instagram posts by GPS coordinate, with async fuzzy location matching against venue/coordinate data. — 2 GitHub stars.
GitHub Actions CI with a real test suite for the Instagram dashboard.
AWS SAM Lambda project with staged dev/staging/production deployment pipeline and Git Flow branching, built via 13 merged PRs on a proper review workflow.
CI pipeline enforcing PyLint and Black formatting via pre-commit hooks. — PyLint score raised to 9.25.
Auto-generated documentation site via MkDocs from docstrings.
Claude skill → AWS Lambda Function URL (Go) → Strava OAuth via Secrets Manager → Strava API; deployed with CDK in Go.
Routes:
/activities,/activities/{id},/activities/{id}/streams,/activities/{id}/laps,/mcp; server-side stream condensing to 200 points.Explored Garmin integration; official API requires partner application, adopted screenshot-based workaround as interim.
Added a second Lambda integrating Intervals.ICU alongside Strava.
Unit, integration, MCP-protocol, and stream-handling test coverage.
Multi-stage data pipeline over personal Spotify streaming history: ScraPy web scraping plus Spotify, GeoDB, and MusicBrainz API integration, with a custom recency-weighting system scoring full streaming history.
Deployed as a Streamlit dashboard on Heroku. — 1 GitHub star.
Series of fastai-based classifiers, each deployed as a Streamlit or Voila app: bird species image classifier, movie recommender (collaborative filtering with a dot-product bias model), COVID mask detector, German handwritten-character recogniser, IMDb review sentiment classifier. — three accompanied by published Medium write-ups.
IMDb review sentiment classifier (RNN/LSTM, FastAI/PyTorch): iteratively debugged and improved architecture, documenting each step.
COVID mask detector: Bing Image Search API scraping for training data, FastAI DataBlock pipeline, deployed to Streamlit Sharing.
Self-directed comparison of SST (Serverless Stack Toolkit) against the Serverless Framework for TypeScript/AWS projects, off the back of prior Serverless Framework use.
Cited SST's Live Lambda Dev (local development with automatic reloading) as the standout productivity gain over Serverless Framework.
Skills
Levelled honestly — production means shipped and owned.
Production
shipped and owned
- Python
- AWS (CDK, Lambda, API Gateway, Kinesis, State Machine, Secrets Manager)
- PostgreSQL
- CI/CD
- IaC
- Testing (unit/integration/E2E)
- Docker
- SQL Server
- AI-assisted development (Claude Code / MCP)
- .NET / C#
Hands-on
built real things, not in production
- FastAPI
- Go
- TypeScript
- Linux / shell scripting
- Observability (OTel, Jaeger, Prometheus, Grafana)
- Geospatial (geohash, GDAL, GeoPandas, DEM)
- Redis / caching design
- GitOps / ArgoCD
- GraphQL
- React
- RAG / vector DB / LLM
- LLM evaluation / token accounting
- MLflow / experiment tracking
Exposure
adjacent, observed
- Kubernetes
- MongoDB
Learning
in progress
- Kafka
Writing
10 published technical articles.
- Making a Geospatial API Fast: Four Techniques, MeasuredTakes a naive FastAPI/Postgres radius search and layers four optimisations onto it — pagination, a composite
(lat, lon)index, geohash prefix search and Redis caching — benchmarking each over the full request path rather than isolated database time, and naming where each one stops helping. - Go vs Python on AWS Lambda: A Geospatial BenchmarkBenchmarks four implementations of a geospatial workload on AWS Lambda across memory tiers. Go wins cold starts outright, but NumPy beats it once warm at high memory, so the conclusion is a decision framework rather than a winner.
- Real-Time Event Analytics: Lessons from Building a Production-Ready AWS PipelineHow an event ingestion pipeline is built on CDK, Lambda, Kinesis Firehose and Athena, and why analytics workloads can trade latency for cost: batching events into S3 and letting partition projection replace crawler runs. Includes the full monthly cost breakdown at demo scale.
- Understanding Transformer Architecture: The Backbone of Modern NLPAn introduction to how NLP architectures evolved into the transformer: attention, multi-head attention and transfer learning, built up from what came before. Closes on the parts that are less often discussed — the carbon cost of training these models, and what interpretability research can and cannot yet tell us about them.
- Optimizing FLAN T5: A Practical Guide to PEFT with LoRA & Soft PromptsA practical guide to fine-tuning FLAN-T5 for dialogue summarisation without the cost of a full fine-tune. LoRA reaches within about a point of full fine-tuning ROUGE-L in a fraction of the training time, with adapters small enough to store per task.
- Shifting Gears: A Journey from Serverless Framework to SSTA side-by-side comparison of SST against the Serverless Framework for TypeScript on AWS, written after using the latter on real projects. Covers where SST's native TypeScript support and prebuilt constructs cut configuration, and why Live Lambda Dev was the feature that actually changed the working day.
- Newspaper Chatbot (BellingChat?)Building a retrieval-augmented chatbot over Bellingcat's investigative journalism archive: scraping the corpus, embedding it into ChromaDB, and choosing between Llama 2 and Mistral-7B for the generation step.
- Movie Review Sentiment Analysis w/ RNNsClassifying IMDb review sentiment, improved one architecture at a time from a basic RNN at 50% to an LSTM at 87%. Includes the step that made things worse, and the diagnosis of why: vanishing gradients in a naive multilayer network.
- Movie Recommender using Collaborative FilteringHow collaborative filtering turns a sparse table of user ratings into recommendations, built as a dot-product model with bias terms and explained from the intuition up.
- Mask Detector w/ FastAI and Streamlit SharingTraining an image classifier to detect face masks and deploying it to Streamlit Sharing. The interesting part came after deployment: the model failed on bearded faces, which were absent from the scraped training data, and fixing that exposed how much the search results had shaped the dataset.
Education
- BEng Computer EngineeringTU Dublin
Final-year project (
RPi_HEMS) built a home energy management system on a Raspberry Pi: reading a power meter over Modbus TCP and hall-effect sensors over I2C, storing readings in InfluxDB, and controlling a Z-Wave smart plug through OpenZWave to shave demand peaks — a full sensor-to-actuator loop with Grafana dashboards over the time-series data. Peak detection was developed against real Eirgrid national demand data in pandas/SciPy before being pointed at live readings. - Machine LearningCoursera
- Deep Learning for CodersFastAI
- Introduction to StatisticsCoursera
- Advanced Statistics for Data ScienceCoursera