1"use strict";(globalThis.webpackChunkportfolio=globalThis.webpackChunkportfolio||[]).push([[5022],{2519(e){e.exports=JSON.parse('{"archive":{"blogPosts":[{"id":"future-of-machine-learning","metadata":{"permalink":"/machine-learning/future-of-machine-learning","source":"@site/machine-learning/2026-07-12-future-of-machine-learning.md","title":"Future of Machine Learning","description":"The future of machine learning will not be defined only by larger models. Much of the progress will come from learning with less supervision, automating model design, running models closer to users and making them accountable when their decisions affect society.{/truncate/}","date":"2026-07-12T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"AI Ethics","permalink":"/machine-learning/tags/ai-ethics"},{"inline":true,"label":"Edge AI","permalink":"/machine-learning/tags/edge-ai"},{"inline":true,"label":"Self-Supervised Learning","permalink":"/machine-learning/tags/self-supervised-learning"},{"inline":true,"label":"Unit 12","permalink":"/machine-learning/tags/unit-12"}],"readingTime":3.48,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"},"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"future-of-machine-learning","title":"Future of Machine Learning","authors":["rossbulat"],"tags":["Machine Learning","AI Ethics","Edge AI","Self-Supervised Learning","Unit 12"],"date":"2026-07-12T00:00:00.000Z"},"unlisted":false,"nextItem":{"title":"Building a MovieLens Recommender with Biased Matrix Factorisation","permalink":"/machine-learning/movielens-biased-matrix-factorisation"}},"content":"The future of machine learning will not be defined only by larger models. Much of the progress will come from learning with less supervision, automating model design, running models closer to users and making them accountable when their decisions affect society.{/*truncate*/}\\n\\n## Learning with Fewer Labels\\n\\nSelf-supervised learning (SSL) creates training signals from unlabelled data, such as predicting a masked image region or matching two augmented views. A representation can then be fine-tuned with a smaller labelled dataset, reducing expensive annotation in fields such as medical imaging and speech (Ericsson *et al.*, 2022). However, unlabelled does not mean unbiased or lawful. Web, sensor and patient data can contain historical discrimination, personal information and rare cases that a downstream test set misses. SSL may shift expenditure from labelling to large-scale pre-training, so teams must evaluate data provenance, transferability and compute cost rather than assuming it is automatically cheaper.\\n\\n## Automating Model Design\\n\\nNeural Architecture Search (NAS) automates choices such as layers, connections and operations. It can discover architectures that outperform hand-designed alternatives and optimise for several metrics, including accuracy, latency or memory. The catch is that the search space, search strategy and performance estimation method all influence the result (Elsken, Metzen and Hutter, 2019). Repeatedly training candidates can be computationally and environmentally expensive, while results may be difficult to reproduce. A development team should therefore compare NAS with strong manual baselines and treat energy use, hardware constraints and search budget as evaluation metrics.\\n\\n## Intelligence at the Edge\\n\\nEdge AI runs inference on, or near, the device producing data. Lower network latency and the ability to keep raw information local are important advantages for autonomous vehicles, wearable medical devices and smart-city sensors (Zhou *et al.*, 2019). Yet small devices impose limits on memory, battery, cooling and model size. Offline operation also complicates security patches, monitoring and rollback. In safety-critical settings, a fast local prediction is valuable only if the wider system accounts for uncertainty, sensor failure and human override. In smart cities, local processing can improve privacy but does not resolve questions about surveillance, consent or unequal monitoring.\\n\\n## Responsible and Sustainable Progress\\n\\nFuture scalability should be judged by useful performance per unit of data, energy and hardware. Professionals need to document carbon-intensive training, reduce model size without compromising accuracy or safety, test demographic and environmental subgroups, and monitor behaviour after deployment. From a legal perspective, the EU AI Act illustrates the move towards risk-based obligations, transparency and protection of health, safety and fundamental rights (European Parliament and Council of the European Union, 2024).\\n\\nOn an SSL or edge-AI project, these responsibilities would be shared across the virtual team. Data engineers would record lineage and consent, while ML engineers would report uncertainty, efficiency and subgroup errors. Domain specialists would define harmful failures for the deployment c
1ontext. Security and MLOps work would cover attacks, drift and recovery, with product and legal colleagues keeping release decisions accountable. Shared model cards, reproducible experiments and explicit release gates would make each contribution visible.\\n\\n## References\\n\\nElsken, T., Metzen, J.H. and Hutter, F. (2019) \u2018Neural architecture search: A survey\u2019, *Journal of Machine Learning Research*, 20(55), pp. 1\u201321. Available at: [https://jmlr.org/papers/v20/18-598.html](https://jmlr.org/papers/v20/18-598.html) (Accessed: 12 July 2026).\\n\\nEricsson, L., Gouk, H., Loy, C.C. and Hospedales, T.M. (2022) \u2018Self-supervised representation learning: Introduction, advances, and challenges\u2019, *IEEE Signal Processing Magazine*, 39(3), pp. 42\u201362. Available at: [https://doi.org/10.1109/MSP.2021.3134634](https://doi.org/10.1109/MSP.2021.3134634) (Accessed: 12 July 2026).\\n\\nEuropean Parliament and Council of the European Union (2024) \u2018Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence\u2019, *Official Journal of the European Union*, 12 July. Available at: [https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng](https://eur-lex.europa.eu/eli/reg/2024/1689/oj/eng) (Accessed: 12 July 2026).\\n\\nZhou, Z., Chen, X., Li, E., Zeng, L., Luo, K. and Zhang, J. (2019) \u2018Edge intelligence: Paving the last mile of artificial intelligence with edge computing\u2019, *Proceedings of the IEEE*, 107(8), pp. 1738\u20131762. Available at: [https://doi.org/10.1109/JPROC.2019.2918951](https://doi.org/10.1109/JPROC.2019.2918951) (Accessed: 12 July 2026)."},{"id":"movielens-biased-matrix-factorisation","metadata":{"permalink":"/machine-learning/movielens-biased-matrix-factorisation","source":"@site/machine-learning/2026-07-11-movielens-biased-matrix-factorisation.md","title":"Building a MovieLens Recommender with Biased Matrix Factorisation","description":"Google Colab notebook Download .ipynb","date":"2026-07-11T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"Recommender Systems","permalink":"/machine-learning/tags/recommender-systems"},{"inline":true,"label":"Matrix Factorisation","permalink":"/machine-learning/tags/matrix-factorisation"},{"inline":true,"label":"SHAP","permalink":"/machine-learning/tags/shap"},{"inline":true,"label":"Unit 12","permalink":"/machine-learning/tags/unit-12"}],"readingTime":5.51,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"},"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"movielens-biased-matrix-factorisation","title":"Building a MovieLens Recommender with Biased Matrix Factorisation","authors":["rossbulat"],"tags":["Machine Learning","Recommender Systems","Matrix Factorisation","SHAP","Unit 12"],"date":"2026-07-11T00:00:00.000Z"},"unlisted":false,"prevItem":{"title":"Future of Machine Learning","permalink":"/machine-learning/future-of-machine-learning"},"nextItem":{"title":"Inside a CNN: Testing Similar and Dissimilar Images","permalink":"/machine-learning/cnn-explainer-image-recognition"}},"content":"**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1SllXn8PqnFDT1SRRRj_sXoxOnsYCkCC1) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-07-11-movielens-biased-matrix-factorisation.ipynb)\\n\\nFor this exercise, I developed a recommender system in stages, from simple baselines to biased matrix factorisation, before asking whether SHAP explanations make individual predictions more understandable. What stood out was that better rating prediction did not automatically produce better top-10 ranking.{/*truncate*/}\\n\\n## Data and Method\\n\\nThe notebook downloads MovieLens 100K directly from GroupLens and verifies the archive checksum before loading it. Validation confirmed 100,000 unique user-movie ratings from 943 users across 1,682 films, with no missing fields and ratings restricted to 1\u20135. MovieLens is deliberately modest, but its stable structure makes it useful for comparing recommender algorithms (Harper and Konstan, 2015).\\n\\nTo avoid training on a user\'s future, I sorted each user\'s ratings chronologically. Their earliest 80% formed training data, the next 10% validation data and the latest 10% test data. This produced 79,619 training, 9,942 validation and 10,439 test ratings. Every test user had training history, although 52 test rows involved movies absent from training. Those rows used a bias-only cold-start fallback.\\n\\nI first calculated the training-set global mean, then fitted regularised user and movie biases. These baselines made it possible to check whether added complexity genuinely improved generalisation rather than merely fitting the training data (Deepchecks, n.d.). The final model estimates a rating as the global mean plus user bias, movie bias and a 20-factor latent interaction. Its parameters were learned with SGD, regularisation and validation monitoring, following the collaborative-filtering progression discussed by Theobald (2024).\\n\\n## Results and Diagnostics\\n\\nThe executed test-set comparison shows how much each modelling stage reduced rating error. Lower values are better for both metrics.\\n\\n<img src=\\"/img/2026-07-11-movielens-biased-matrix-factorisation/rating-metrics-comparison.png\\" alt=\\"Executed notebook output comparing test RMSE and MAE: biased matrix factorisation scores 0.9875 and 0.7850, the bias model 1.0169 and 0.8124, and the global mean 1.2322 and 1.0255\\" width=\\"300\\" style={{ display: \'block\', margin: \'2rem auto\' }} />\\n\\nMatrix factorisation improved RMSE by about 2.9% over the strong bias baseline. Its known-user/known-movie RMSE was 0.9857, while the cold-movie fallback reached 1.3005. This gap makes the uncertainty around unseen items visible rather than hiding it inside one aggregate score. Test residuals had a mean of \u22120.0613, indicating slight under-prediction overall.\\n\\nRanking told a less flattering story. I treated ratings of 4 or 5 as relevant, excluded movies already seen before the test period, and ranked the remaining catalogue for 856 evaluable users. The macro-averaged comparison at `K=10` is shown below; higher values are better.\\n\\n<img src=\\"/img/2026-07-11-movielens-biased-matrix-factorisation/ranking-metrics-k10-comparison.png\\" alt=\\"Executed notebook output comparing top-10 ranking metrics: the bias model scores 0.0278 precision, 0.0434 recall and 0.0360 NDCG, while biased matrix factorisation scores 0.0263, 0.0376 and 0.0340\\" width=\\"450\\" style={{ display: \'block\', margin: \'2rem auto\' }} />\\n\\nThe bias model led on precision, recall and NDCG. Matrix factorisation therefore predicted rating values more accurately but did not improve the recommendation ordering. This result shows why rating prediction and recommendation ranking must be evaluated separately: a model optimised for squared rating error is not necessarily optimised for top-K discovery, and an unrated film does not necessarily indicate a negative preference.\\n\\n## Does the SHAP Explanation Make Sense?\\n\\nThe latent vectors are difficult to describe to a user. I therefore exposed the matrix-factorisation model\'s three raw-score components: user bias, movie bias and latent match. They reconstructed the unclipped score exactly, with an R\xb2 of 1.0 and MAE of 0. SHAP then expressed how each component moved a prediction away from the background value, consistent with its additive local-explanation design (Lundberg and Lee, 2017).\\n\\nFor user 632, the model predicted **4.60** for *Star Wars (1977)*, whose training mean was 4.35. Movie bias contributed +0.695 and latent match +0.565, while user bias contributed \u22120.199. For the same user, *Leave It to Beaver (1997)* was predicted **1.60**; its movie bias contributed \u22121.770 and dominated the explanation. The contrast makes sense as a description of the model and training data. It does not prove that either film is objectively good or that these factors caused the user\'s preference. The latent-match term remains abstract, and SHAP values depend on the chosen background. Rahim M and Basheer K.P (2025) likewise distinguish local explanations for individual recommendations from aggregated global explanations and stress translating technical attributions for end users.\\n\\n## Critical Reflection\\n\\nWorking in stages kept three questions separate: whether matrix factorisation improved held-out rating prediction, whether it ranked relevant unseen f
1ilms effectively, and whether SHAP could communicate its scores without implying causation. This prevented the stronger RMSE from obscuring the weaker ranking performance.\\n\\nThe chronological split is more deployment-like than a random split, but one historical MovieLens subset cannot establish current performance. Ranking metrics also rely on the unobserved-feedback assumption noted above, while a plausible SHAP explanation remains a diagnostic rather than proof that the model is correct.\\n\\nIn a development team, data engineers would own validation and lineage, ML engineers the reproducible experiment, product specialists the definition of relevance and error costs, and QA/MLOps colleagues cold-start and drift testing. My role would include reporting the weaker ranking result and explanation limitations alongside the stronger RMSE. Results, limitations and component fidelity should be documented so stakeholders can challenge the recommendations rather than receiving only the model\'s most favourable metric.\\n\\n## References\\n\\nDeepchecks (n.d.) \u2018Baseline models\u2019. Available at: [https://deepchecks.com/glossary/baseline-models/](https://deepchecks.com/glossary/baseline-models/
1) (Accessed: 11 July 2026).\\n\\nHarper, F.M. and Konstan, J.A. (2015) \u2018The MovieLens datasets: History and context\u2019, *ACM Transactions on Interactive Intelligent Systems*, 5(4), article 19. Available at: [https://doi.org/10.1145/2827872](https://doi.org/10.1145/2827872) (Accessed: 11 July 2026).\\n\\nLundberg, S.M. and Lee, S.-I. (2017) \u2018A unified approach to interpreting model predictions\u2019, *Advances in Neural Information Processing Systems*, 30. Available at: [https://proceedings.neurips.cc/paper_files/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html](https://proceedings.neurips.cc/paper_files/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html) (Accessed: 11 July 2026).\\n\\nRahim M, N. and Basheer K.P, M. (2025) \u2018A hybrid explainability framework for recommender systems using SHAP with natural language interpretations\u2019, *International Journal of Innovative Research and Scientific Studies*, 8(6), pp. 687\u2013703. Available at: [https://doi.org/10.53894/ijirss.v8i6.9669](https://doi.org/10.53894/ijirss.v8i6.9669) (Accessed: 11 July 2026).\\n\\nTheobald, O. (2024) *Machine Learning: Make Your Own Recommender System*. Birmingham: Packt Publishing. Available at: [https://www.oreilly.com/library/view/machine-learning-make/9781835882061/](https://www.oreilly.com/library/view/machine-learning-make/9781835882061/) (Accessed: 11 July 2026)."},{"id":"cnn-explainer-image-recognition","metadata":{"permalink":"/machine-learning/cnn-explainer-image-recognition","source":"@site/machine-learning/2026-07-08-cnn-explainer-image-recognition.md","title":"Inside a CNN: Testing Similar and Dissimilar Images","description":"I used CNN Explainer to follow two images through a small convolutional neural network a model can only choose from the labels and examples represented in its dataset.{/truncate/}","date":"2026-07-08T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"CNNs","permalink":"/machine-learning/tags/cn-ns"},{"inline":true,"label":"Computer Vision","permalink":"/machine-learning/tags/computer-vision"},{"inline":true,"label":"Image Classification","permalink":"/machine-learning/tags/image-classification"},{"inline":true,"label":"Unit 10","permalink":"/machine-learning/tags/unit-10"}],"readingTime":4.99,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"},"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"cnn-explainer-image-recognition","title":"Inside a CNN: Testing Similar and Dissimilar Images","authors":["rossbulat"],"tags":["Machine Learning","CNNs","Computer Vision","Image Classification","Unit 10"],"date":"2026-07-08T00:00:00.000Z"},"unlisted":false,"prevItem":{"title":"Building a MovieLens Recommender with Biased Matrix Factorisation","permalink":"/machine-learning/movielens-biased-matrix-factorisation"},"nextItem":{"title":"Collaborative Discussion 2 Reflection","permalink":"/machine-learning/collab-2-reflection"}},"content":"I used [CNN Explainer](https://poloclub.github.io/cnn-explainer/) to follow two images through a small convolutional neural network: an espresso, which matches one of the model\'s trained classes, and a sunflower, which does not. Comparing them exposed a basic limitation of image classifiers: a model can only choose from the labels and examples represented in its dataset.{/*truncate*/}\\n\\n## The Activity\\n\\nCNN Explainer visualises a pretrained Tiny VGG network in the browser. It accepts a 64 \xd7 64 RGB image and shows the activation maps produced by convolution, ReLU and max-pooling layers before flattening the learned features and applying softmax across ten output classes.\\n\\n\\n\\n*The full network. Each column is a layer and each small square is an activation map.*\\n\\nI first explored the supplied examples to understand the interface. I then used two public-domain photographs from Wikimedia Commons:\\n\\n- [Espresso 01.jpg](https://commons.wikimedia.org/wiki/File:Espresso_01.jpg), photographed by Calvin Ballantine and released into the public domain.\\n- [Sunflower.JPG](https://commons.wikimedia.org/wiki/File:Sunflower.JPG), photographed by Adam Monkhouse and released into the public domain.\\n\\nThe espresso is a **similar** image because `espresso` is one of the model\'s output classes. The sunflower is deliberately **dissimilar**: there is no `sunflower`, `flower` or general `plant` class among the ten possible answers.\\n\\n## What Changed Across the Layers?\\n\\nThe early convolutional maps still resembled the input. On the espresso image, different filters emp
1hasised the rim of the cup, the saucer, shadows and changes in brightness. These are local features rather than a complete concept of \u201ccoffee\u201d.\\n\\nThe ReLU layer applied `max(0, x)` element by element. Positive responses remained visible, while negative responses were replaced with zero. In the selected example below, an input activation of `-0.2` therefore produced an output of `0`. This sparsity helps the network retain useful responses and introduces the non-linearity needed to learn more complex patterns.\\n\\n\\n\\n*An early ReLU activation: negative values are suppressed while positive filter responses are retained.*\\n\\nAs the image travelled deeper into the network, the activation maps became smaller and less recognisable. Max-pooling reduced their spatial size by keeping the strongest response in each local region. Later convolutional layers combined simple edges and colour changes into more abstract arrangements. The final pooling layer produced ten 13 \xd7 13 maps, which were flattened into a vector of 1,690 values for classification.\\n\\n## Similar Image: Espresso\\n\\nThe uploaded espresso produced a strong and coherent path through the network. Several early maps clearly highlighted the circular cup and saucer, and later maps responded to more selective combinations of these features.\\n\\n\\n\\n*The uploaded espresso moving through convolution, ReLU and pooling layers.*\\n\\nThe final softmax output assigned **93.49%** to `espresso`. The next-highest score was only about **3.58%** for `ladybug`. This result makes sense because the object belongs to a trained class and shares useful visual characteristics with the training examples.\\n\\n\\n\\n*Flattened features feed the softmax output, which scores the uploaded image as espresso at 0.9349.*\\n\\n## Dissimilar Image: Sunflower\\n\\nThe sunflower created strong circular and radial activation patterns, especially around the flower head. Those patterns survived into deeper layers even though the network had never been given a sunflower label.\\n\\n\\n\\n*The sunflower also produces meaningful activations, but the correct semantic class is unavailable.*\\n\\nSoftmax then had to distribute all probability across the ten known classes. It returned **72.82%** for `bell pepper`, approximately **14.36%** for `orange`, and **12.74%** for `ladybug`. The radial shape, warm colour and central region may resemble features the network learned for those classes, but `bell pepper` is not a meaningful description of the image.\\n\\n\\n\\n*The dissimilar image is confidently forced into the nearest available class: bell pepper at 0.7282.*\\n\\n## Dataset Applicability and Challenges\\n\\nThe experiment showed why dataset design is just as critical as model architecture. A compact ten-class dataset makes the network fast enough to inspect interactively and is useful for learning how layers transform an image. It is not representative of open-world image recognition, where an input may contain one object or several at once.\\n\\nThe espresso result shows that CNNs can generalise beyond an exact training photograph when the new image is similar to the training distribution. However, the sunflower exposes a different problem: softmax confidence is conditional on the available labels. It answers \u201cwhich of these ten classes is most likely?\u201d, not \u201cis this actually one of these ten classes?\u201d. A high score can therefore be confidently wrong when the dataset omits the true class.\\n\\nOther challenges include limited variation in viewpoint, scale, background and lighting; class imbalance; low-resolution inputs; and hidden bias in how training images were collected. A stronger evaluation would use many images per condition, include out-of-distribution examples, report top-k predictions and calibration, and allow the system to reject an image as `unknown` when no class is suitable.\\n\\n## Reflection\\n\\nLooking inside the intermediate layers made the classification more understandable. Early layers detected local visual patterns, ReLU removed negative responses, pooling compressed the representation and later layers assembled
1increasingly abstract evidence. However, none of those operations can recover a label that is missing from the training task. The model\'s usefulness is limited by both the quality of its learned features and the scope of its dataset.\\n\\n## References\\n\\n- Wang, Z.J. *et al.* (2020) [CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization](https://arxiv.org/abs/2004.15004).\\n- Polo Club of Data Science (2020) [CNN Explainer](https://poloclub.github.io/cnn-explainer/).\\n- Wang, J. (2020) [CNN Explainer demo video](https://www.youtube.com/watch?v=HnWIHWFbuUQ)."},{"id":"collab-2-reflection","metadata":{"permalink":"/machine-learning/collab-2-reflection","source":"@site/machine-learning/2026-07-07-collab-2-reflection.md","title":"Collaborative Discussion 2 Reflection","description":"This reflection covers my initial post, peer responses, feedback and summary for the second collaborative discussion in the Machine Learning module, which focused on AI writers, human oversight and responsible use.{/ truncate /}","date":"2026-07-07T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"MSc","permalink":"/machine-learning/tags/m-sc"},{"inline":true,"label":"Reflection","permalink":"/machine-learning/tags/reflection"},{"inline":true,"label":"Unit 8","permalink":"/machine-learning/tags/unit-8"},{"inline":true,"label":"Unit 9","permalink":"/machine-learning/tags/unit-9"},{"inline":true,"label":"Unit 10","permalink":"/machine-learning/tags/unit-10"}],"readingTime":4.36,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"},"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"collab-2-reflection","title":"Collaborative Discussion 2 Reflection","authors":["rossbulat"],"tags":["Machine Learning","MSc","Reflection","Unit 8","Unit 9","Unit 10"]},"unlisted":false,"prevItem":{"title":"Inside a CNN: Testing Similar and Dissimilar Images","permalink":"/machine-learning/cnn-explainer-image-recognition"},"nextItem":{"title":"Model Performance Parameter Experiment","permalink":"/machine-learning/model-performance-parameter-experiment"}},"content":"This reflection covers my initial post, peer responses, feedback and summary for the second collaborative discussion in the Machine Learning module, which focused on AI writers, human oversight and responsible use.{/* truncate */}\\n\\n## Initial Post Reflection\\n\\nThe risk-gradient framing gave my initial post a useful structure. I separated low-stakes administrative writing from high-stakes legal, medical, HR, educational and public-facing uses, then treated creative writing as a separate cultural problem. Hutson helped establish the paradox that fluent prose can look authoritative without grounded understanding, while Noy and Zhang helped me acknowledge the real productivity gains.\\n\\nHowever, the post treated these governance controls as more straightforward than they are. I named controls such as approved data use, source verification, bias testing, audit logs, human sign-off and accountability, but did not explain how they should work. \\"Human sign-off\\" can be weak if the reviewer is rushed, lacks expertise or assumes the model is correct.\\n\\nI also could have been more critical of administrative use. Routine summaries, emails and report drafts can still contain sensitive data or introduce false details that later become part of an official record.\\n\\n## Exchange with Abdullah\\n\\nAbdullah\'s response challenged an assumption I had not examined closely. He accepted the need for human oversight but pointed out that human-in-the-loop governance can become a procedural formality if reviewers are affected by automation bias or are not trained to intervene. I should have been more specific about what effective oversight involves, including escalation thresholds, reviewer competence, evidence checks, auditability and permission to override model output.\\n\\nHis point about diversity-aware prompting also strengthened the creative-writing argument. I had cited the risk that AI assistance may reduce collective diversity, but said little about prevention beyond keeping human judgement central. I could have explored varied prompts, multiple models, deliberate constraints, or human editorial standards as possible safeguards.\\n\\n## Peer Response Reflection: Ariel\'s Post\\n\\nWith Ariel\'s post, I was able to make the question more precise. Ariel focused on simulated cognition, agentic AI and the possibility that users may outsource cognitive work to
1systems that do not actually reason. I separated the question of whether an LLM \\"thinks\\" from the reliability of its outputs.\\n\\nQuestioning the social-media analogy was also important. Evidence about habitual platform checking and adolescent brain development does not automatically transfer to LLM-assisted reasoning, so the comparison should not be treated as established evidence.\\n\\nHowever, my response narrowed the issue too quickly. Cognitive outsourcing is difficult to evidence, but that does not make it unimportant. I should have asked what evidence would be needed and engaged more with Ariel\'s concern about task execution and decision support.\\n\\n## Peer Response Reflection: Paul\'s Post\\n\\nPaul gave strong examples of LLM benefits and risks, including nonsensical outputs, dangerous advice, opaque complexity and harmful stereotypes. My response built on these examples by distinguishing simulated empathy from real empathy. An empathetic tone can be useful in customer-service drafting, but dangerous in safeguarding or healthcare contexts.\\n\\nI also challenged Paul\'s statement that there is \\"no effective solution\\" to bias and harmful output. Accepting that claim would have been too fatalistic. Dataset documentation, curation, bias testing, disclosure, source checking and accountable human approval may not eliminate harm, but they can reduce it.\\n\\nWhere the response fell short was in repeating ideas from my initial post rather than developing them further. I could have examined limited interpretability more closely: if model behaviour cannot be fully explained, what level of testing, monitoring or restriction is acceptable before deployment in sensitive domains? I also should have treated the self-harm example separately from general high-stakes use, because safeguarding interactions require much higher standards for safety and escalation.\\n\\n## Summary Post Reflection\\n\\nBy the summary post, my position had developed. I moved away from asking whether AI writers are useful and focused instead on the conditions under which they can be used responsibly. The distinction between AI as a writing aid and AI as a source of authority was clearer.\\n\\nMy treatment of human-in-the-loop review had also developed. I no longer presented human sign-off as sufficient by itself, instead linking oversight to escalation criteria, source verification, audit trails, bias testing and periodic auditing. The summary handled creative writing more carefully too, acknowledging the tension between democratising expression and narrowing collective diversity.\\n\\nEven so, the summary still named governance mechanisms more readily than it explored trade-offs such as cost, speed, expertise and organisational incentives. It could also have made a stronger connection to machine learning evaluation through prompt testing, output monitoring, and failure-case logging.\\n\\n## Overall Reflection\\n\\nAcross the discussion, I kept the argument balanced and evidence-led, avoiding the view that AI writers are either harmless productivity tools or inherently unacceptable systems. However, I relied too heavily on governance language without always translating it into practice. I identified safeguards more consistently than I explained how they would be implemented or tested.\\n\\nWhat I will carry forward is that responsible AI goes beyond simply \\"keeping a human in the loop\\". The real test is whether that person has the expertise, evidence, authority and time to intervene effectively, especially under pressure from speed, cost, convenience and overconfidence in fluent machine output."},{"id":"model-performance-parameter-experiment","metadata":{"permalink":"/machine-learning/model-performance-parameter-experiment","source":"@site/machine-learning/2026-06-28-model-performance-writeup.md","title":"Model Performance Parameter Experiment","description":"I compared classification and regression parameter choices using held-out test data.{/truncate/}","date":"2026-06-28T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"Model Evaluation","permalink":"/machine-learning/tags/model-evaluation"},{"inline":true,"label":"Classification","permalink":"/machine-learning/tags/classification"},{"inline":true,"label":"Regression","permalink":"/machine-learning/tags/regression"},{"inline":true,"label":"Unit 11","permalink":"/machine-learning/tags/unit-11"}],"readingTime":1.78,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"}
1,"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"model-performance-parameter-experiment","title":"Model Performance Parameter Experiment","authors":["rossbulat"],"tags":["Machine Learning","Model Evaluation","Classification","Regression","Unit 11"]},"unlisted":false,"prevItem":{"title":"Collaborative Discussion 2 Reflection","permalink":"/machine-learning/collab-2-reflection"},"nextItem":{"title":"CNN Object-Recognition Notebook: Findings","permalink":"/machine-learning/cnn-object-recognition"}},"content":"I compared classification and regression parameter choices using held-out test data.{/*truncate*/}\\n\\n**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1QrsSGlcRq-mE59SjnNJc2D9oy2GDXQrc) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-06-28-model-performance-writeup.ipynb)\\n\\n## Method and Findings\\n\\nI ran the supplied notebook and added two reproducible parameter experiments, both with a fixed random seed and held-out test data. For classification, I used the breast-cancer dataset with a standardised radial-basis-function support vector classifier. I varied `C` (`0.1`, `1`, `10`, `100`) and `gamma` (`0.001`, `0.01`, `0.1`, and `scale`) and measured ROC AUC. The best test AUC was **0.9987** at `C=10` and `gamma=0.01`. More model flexibility did not always improve generalisation: at `C=100` and `gamma=0.1`, training AUC reached **1.0000**, but test AUC fell to **0.9901**, indicating overfitting.\\n\\nFor regression, I used the diabetes dataset with a random-forest regressor. I varied `n_estimators` (`25`, `100`, `300`) and `max_depth` (`2`, `4`, `8`, and unlimited). The best test R\xb2 was **0.4995**, with **300 trees** and `max_depth=4`; its test RMSE was **52.61**. Deeper trees achieved much higher training R\xb2 but weaker test performance. For example, 25 trees with depth 8 produced a training R\xb2 of **0.8662** but a test R\xb2 of **0.4201**. This generalisation gap is why parameter choices should be based on unseen data rather than training scores. R\xb2 is a score rather than an error, so higher values are better; RMSE is an error measure, so lower values are better.\\n\\n## Legal, Social, Ethical and Professional Reflection\\n\\nA strong overall metric does not prove that a model is safe or fair. AUC can hide performance differences between demographic groups and does not select a decision threshold; false positives and false negatives may have unequal consequences, especially in health-related contexts.\\n\\nMachine-learning professionals should use data collected with appropriate permission, protect confidential information, document limitations and avoid presenting a classroom model as clinically valid. They should also test performance across subgroups, consider accessibility and unequal social impacts, keep reproducible records, and communicate uncertainty honestly. In practice, metrics need to sit alongside governance, human review, security, monitoring and a clearly defined purpose."},{"id":"cnn-object-recognition","metadata":{"permalink":"/machine-learning/cnn-object-recognition","source":"@site/machine-learning/2026-06-21-cnn-object-recognition.md","title":"CNN Object-Recognition Notebook: Findings","description":"My CNN object-recognition notebook covers the model setup, training results, a test prediction and the limitations of using CIFAR-10 for image classification.{/truncate/}","date":"2026-06-21T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"CNNs","permalink":"/machine-learning/tags/cn-ns"},{"inline":true,"label":"Computer Vision","permalink":"/machine-learning/tags/computer-vision"},{"inline":true,"label":"CIFAR-10","permalink":"/machine-learning/tags/cifar-10"},{"inline":true,"label":"Unit 10","permalink":"/machine-learning/tags/unit-10"}],"readingTime":1.66,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"}
1,"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"cnn-object-recognition","title":"CNN Object-Recognition Notebook: Findings","authors":["rossbulat"],"tags":["Machine Learning","CNNs","Computer Vision","CIFAR-10","Unit 10"],"date":"2026-06-21T00:00:00.000Z"},"unlisted":false,"prevItem":{"title":"Model Performance Parameter Experiment","permalink":"/machine-learning/model-performance-parameter-experiment"},"nextItem":{"title":"Facial Recognition, CNNs and the Cost of Being Wrong","permalink":"/machine-learning/facial-recognition-ethics"}},"content":"My CNN object-recognition notebook covers the model setup, training results, a test prediction and the limitations of using CIFAR-10 for image classification.{/*truncate*/}\\n\\n**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1DEVDEt95Geg5DFcl-1C-87ZgBMN3GjcB) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-06-21-cnn-object-recognition.ipynb)\\n\\nThe notebook trains a convolutional neural network on CIFAR-10, a balanced dataset of 32\xd732 colour images across ten classes. I retained the original architecture: two convolution-and-pooling blocks, a 256-unit dense layer and a ten-class softmax output. The model has 225,610 trainable parameters. The data was divided into 40,000 training images, 10,000 validation images and 10,000 test images, with pixel values normalised to `float32` values between zero and one.\\n\\nEarly stopping ended training after seven epochs. Training accuracy rose to 73.25%, while validation accuracy reached 63.61%. Validation loss was lowest at epoch five (1.0692) and then increased for two epochs, even as training performance continued to improve. This divergence suggests overfitting. On the unseen test set, the model achieved 63.58% accuracy with a loss of 1.1105.\\n\\nFor the required single-image experiment, I changed the selected test image from index 16 to index 12. The true class was **dog**, and the model also predicted **dog**, so the prediction was correct. However, its confidence was only 34.59%, leaving substantial uncertainty despite the correct result.\\n\\nPerformance also varied by class. Automobile recall was 0.80, whereas cat precision was only 0.39. The confusion matrix showed considerable overlap between visually similar animal classes: 266 dogs were classified as cats, while 163 cats were classified as dogs. CIFAR-10 is useful for comparing algorithms because it is small, balanced and quick to train, but its low resolution removes detail and its ten broad categories do not represent the variety of real deployment data. A correct result on one image therefore cannot establish reliability. Data augmentation, regularisation, restoring the best validation weights and testing on a more representative dataset would provide a stronger evaluation."},{"id":"facial-recognition-ethics","metadata":{"permalink":"/machine-learning/facial-recognition-ethics","source":"@site/machine-learning/2026-06-20-facial-recognition-ethics.md","title":"Facial Recognition, CNNs and the Cost of Being Wrong","description":"Wall\'s article made me think about facial recognition less as a model-accuracy problem and more as a deployment problem: what happens when an organisation treats a CNN score as objective evidence? A convolutional neural network can return a probability or similarity score, but that score does not, by itself, establish a person\u2019s identity. In a photo app, a wrong match can merely be irritating; but in policing, border control, or military use, it can expose someone to surveillance, exclusion, arrest, or harm (Wall, 2019).{/truncate/}","date":"2026-06-20T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"Ethics","permalink":"/machine-learning/tags/ethics"},{"inline":true,"label":"CNNs","permalink":"/machine-learning/tags/cn-ns"},{"inline":true,"label":"Facial Recognition","permalink":"/machine-learning/tags/facial-recognition"},{"inline":true,"label":"Unit 9","permalink":"/machine-learning/tags/unit-9"}],"readingTime":2.42,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"}
1,"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"facial-recognition-ethics","title":"Facial Recognition, CNNs and the Cost of Being Wrong","authors":["rossbulat"],"tags":["Machine Learning","Ethics","CNNs","Facial Recognition","Unit 9"]},"unlisted":false,"prevItem":{"title":"CNN Object-Recognition Notebook: Findings","permalink":"/machine-learning/cnn-object-recognition"},"nextItem":{"title":"Model Comparison, Explainability and Trust in Computer Vision","permalink":"/machine-learning/model-comparison-explainability-trust"}},"content":"Wall\'s article made me think about facial recognition less as a model-accuracy problem and more as a deployment problem: what happens when an organisation treats a CNN score as objective evidence? A convolutional neural network can return a probability or similarity score, but that score does not, by itself, establish a person\u2019s identity. In a photo app, a wrong match can merely be irritating; but in policing, border control, or military use, it can expose someone to surveillance, exclusion, arrest, or harm (Wall, 2019).{/*truncate*/}\\n\\n## Bias and False Positives\\n\\nThe main ethical issue for me is the asymmetry of false positives. If a system wrongly identifies someone, that person carries the consequences, while the organisation using the system may frame it as a technical error. This is especially serious when the affected person cannot inspect the model, challenge the evidence, or understand why they were matched.\\n\\nWall (2019) also discusses performance differences across skin tone and gender. That made me more cautious about aggregate accuracy. A model can look strong overall while failing more often for specific groups. If training data over-represents white men, or reflects unequal policing, the CNN may reproduce social bias with a technical label.\\n\\n## Social and Legal Consequences\\n\\nPublic facial-recognition systems can identify and record people as they travel, meet others or attend political events, often without their knowledge or consent. Knowing that this monitoring is possible may discourage people from participating in lawful protests or associating with particular groups. It can also reinforce existing policing patterns: communities under greater surveillance generate more recorded encounters, and those records may then be used to justify even more surveillance in the same communities.\\n\\nLegally, this raises questions about privacy, consent, proportionality, data retention, and appeal. A \\"human in the loop\\" is not enough if the reviewer is likely to defer to the system score. Real safeguards need trained reviewers, clear thresholds, audit records, and a challenge process that ordinary people can use.\\n\\n## Professional Accountability\\n\\nFor me, deployment is a part of engineering responsibility. Machine-learning teams should ask where the data came from, whose faces are under-represented, which groups have higher error rates, and what happens after a match. They should document limitations, monitor systems after release, and support independent audits. Some use cases may need flat-out refusal to deploy, or at least a moratorium until the social and legal issues are addressed.\\n\\n## Reflection\\n\\nMy view is that facial recognition may be defensible in narrow, controlled settings, but mass public surveillance is different. The question is not only *can the CNN recognise a face?* It is also *should it be used here, who carries the risk when it fails, and who is accountable?*\\n\\n## Reference\\n\\nWall, M. (2019) \'Biased and wrong? Facial recognition tech in the dock\', *BBC News*, 4 July. Available at: [https://www.bbc.com/news/business-48842750](https://www.bbc.com/news/business-48842750) (Accessed: 20 June 2026)."},{"id":"model-comparison-explainability-trust","metadata":{"permalink":"/machine-learning/model-comparison-explainability-trust","source":"@site/machine-learning/2026-06-10-model-comparison-explainability-trust.md","title":"Model Comparison, Explainability and Trust in Computer Vision","description":"Using Yu et al. (2024) as a comparison framework, I consider how conventional machine learning and deep learning could be evaluated for production-line visual inspection, and how their decisions could be explained to the people expected to act on them.{/truncate/}","date":"2026-06-10T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"Deep Learning","permalink":"/machine-learning/tags/deep-learning"},{"inline":true,"label":"Computer Vision","permalink":"/machine-learning/tags/computer-vision"},{"inline":true,"label":"Explainable AI","permalink":"/machine-learning/tags/explainable-ai"},{"inline":true,"label":"Unit 8","permalink":"/machine-learning/tags/unit-8"}],"readingTime":5.32,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"}
1,"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"model-comparison-explainability-trust","title":"Model Comparison, Explainability and Trust in Computer Vision","authors":["rossbulat"],"tags":["Machine Learning","Deep Learning","Computer Vision","Explainable AI","Unit 8"],"date":"2026-06-10T00:00:00.000Z"},"unlisted":false,"prevItem":{"title":"Facial Recognition, CNNs and the Cost of Being Wrong","permalink":"/machine-learning/facial-recognition-ethics"},"nextItem":{"title":"Gradient Descent Learning Rate and Iterations","permalink":"/machine-learning/gradient-descent-learning-rate-iterations"}},"content":"Using Yu *et al.* (2024) as a comparison framework, I consider how conventional machine learning and deep learning could be evaluated for production-line visual inspection, and how their decisions could be explained to the people expected to act on them.{/*truncate*/}\\n\\n## 1. Model Comparison in Practice: Visual Defect Inspection\\n\\nMy selected domain is **automated visual quality inspection**, specifically estimating surface defects from production-line images to decide whether an item should pass, be reworked or receive manual inspection. A conventional pipeline might extract colour, edge, texture and defect-area measurements for a random forest, support vector machine, multilayer perceptron or decision tree. A CNN instead learns spatial features directly from pixels, which suits the local relationships among edges, textures and shapes (Mienye and Swart, 2024).\\n\\nYu *et al.* (2024) solve a different but transferable regression problem: predicting supply-chain management distribution cost (SCMDC) from 12 order and shipping variables. Their 180,519 records are divided into 70% training, 15% testing and 15% validation data. Five model families are evaluated on the same target, partitions and error measures. The labelled test results in Table 2 are:\\n\\n| Model | RMSE | R\xb2 |\\n| --- | ---: | ---: |\\n| CNN | **0.518** | **0.961** |\\n| Decision tree | 1.372 | 0.896 |\\n| Random forest | 1.738 | 0.892 |\\n| SVM | 1.815 | 0.887 |\\n| MLP | 1.874 | 0.885 |\\n\\nLower RMSE indicates predictions closer to observed costs, while higher R\xb2 indicates more explained variance. The paper also reports a narrower CNN test-error range (\u22126.8 to 10.3). However, its abstract and conclusion describe RMSE 0.528 and R\xb2 0.953 as test results, although the table labels them as training results. I therefore use the explicitly labelled test table above.\\n\\nCNNs learn visual hierarchies without predefined features but often require more labelled images, specialist hardware, training time and energy. Traditional models can compete on smaller datasets, run cheaply on CPUs and expose engineered inputs; a shallow tree is especially transparent. Yet feature engineering may discard information and transfer poorly when lighting, cameras or materials change. Deployment efficiency should therefore always be measured.\\n\\nFor a defensible experiment, I would use the same batch- and time-separated test set for every model. I would report RMSE and R\xb2 alongside errors by defect and product type, then measure latency, memory use and energy consumption. Batch separation prevents nearly identical items leaking across random splits. Because Yu *et al.* use tabular rather than image data, their findings offer a comparison method to reproduce in a multitude of visual-inspection scenarios, but not a direct performance benchmark.\\n\\n## 2. Explainability and Trust in an Inspection System\\n\\nBecause the model informs whether an item is accepted, reworked, sent for further inspection or scrapped\u2014and may even trigger a production-line stop when defects cluster\u2014its recommendations have material consequences. Incorrectly rejecting an item wastes a usable product, while overlooking a defect may allow an unsafe product to reach customers and create warranty costs. Operators therefore need sufficient evidence to understand, question and, when necessary, override its recommendations.\\n\\nAn unexplained model may react to glare, a camera border or a supplier mark rather than the defect, or degrade after lighting and material changes. If rejections cannot be challenged, staff may defer to what looks like an objective score and accountability becomes less clear. Explanations can expose these failures and unequal errors across products, suppliers or sites. The goal is **appropriate trust**: users should see evidence and uncertainty.\\n\\nSHAP attributes feature contributions locally and can aggregate them into global patterns (Lundberg and Lee, 2017); in a conventional pipeline, it might show that defect area and texture raised predicted severity. Image SHAP depends on masking and background choices. LIME highlights influential superpixels through a local surrogate, but its stability should be tested across runs and segmentations (Ribeiro, Singh and Guestrin, 2016). For CNNs, Grad-CAM maps influential regions without retraining (Selvaraju *et al.*, 2017). A map may overlap a crack, but a plausible heatmap is neither causal evidence nor proof of correctness. Each method must be used with its purpose and limitations in mind (Molnar, 2023).\\n\\nI would treat explainability as a product and governance requirement. Quality leads would define error costs and escalation thresholds, while inspectors would check whether highlighted regi
1ons were meaningful. Data engineers would maintain provenance and labelling guidance, and computer-vision engineers would benchmark a simple model. UX and safety specialists would test displays and probe failures. MLOps engineers would monitor drift by camera, site and product type while retaining auditable versions and decision logs.\\n\\nMy responsibility would be to communicate subgroup errors and limitations plainly, preserve reproducible evidence and never present an explanation as certainty. Release should require user testing, stability checks, a documented human override and feedback for disputed cases. Quantitative evidence, visual explanation and accountable review together are stronger evidence than vague measures like a black-box score or an attractive heatmap alone.\\n\\n## References\\n\\nLundberg, S.M. and Lee, S.-I. (2017) \u2018A unified approach to interpreting model predictions\u2019, *Advances in Neural Information Processing Systems*, 30. Available at: [https://proceedings.neurips.cc/paper_files/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html](https://proceedings.neurips.cc/paper_files/paper/2017/hash/8a20a8621978632d76c43dfd28b67767-Abstract.html) (Accessed: 10 June 2026).\\n\\nMienye, I.D. and Swart, T.G. (2024) \u2018A comprehensive review of deep learning: architectures, recent advances, and applications\u2019, *Information*, 15(12), article 755. Available at: [https://doi.org/10.3390/info15120755](https://doi.org/10.3390/info15120755) (Accessed: 10 June 2026).\\n\\nMolnar, C. (2023) *Interpretable Machine Learning: A Guide for Making Black Box Models Explainable*. 2nd edn. Available at: [https://christophm.github.io/interpretable-ml-book/](https://christophm.github.io/interpretable-ml-book/) (Accessed: 10 June 2026).\\n\\nRibeiro, M.T., Singh, S. and Guestrin, C. (2016) \u2018\u201cWhy should I trust you?\u201d: Explaining the predictions of any classifier\u2019, *Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining*, pp. 1135\u20131144. Available at: [https://doi.org/10.1145/2939672.2939778](https://doi.org/10.1145/2939672.2939778) (Accessed: 10 June 2026).\\n\\nSelvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D. and Batra, D. (2017) \u2018Grad-CAM: Visual explanations from deep networks via gradient-based localization\u2019, *Proceedings of the IEEE International Conference on Computer Vision*, pp. 618\u2013626. Available at: [https://openaccess.thecvf.com/content_iccv_2017/html/Selvaraju_Grad-CAM_Visual_Explanations_ICCV_2017_paper.html](https://openaccess.thecvf.com/content_iccv_2017/html/Selvaraju_Grad-CAM_Visual_Explanations_ICCV_2017_paper.html) (Accessed: 10 June 2026).\\n\\nYu, X., Tang, L., Long, L. and Sina, M. (2024) \u2018Comparison of deep and conventional machine learning models for prediction of one supply chain management distribution cost\u2019, *Scientific Reports*, 14, article 24195. Available at: [https://doi.org/10.1038/s41598-024-75114-9](https://doi.org/10.1038/s41598-024-75114-9) (Accessed: 10 June 2026)."},{"id":"gradient-descent-learning-rate-iterations","metadata":{"permalink":"/machine-learning/gradient-descent-learning-rate-iterations","source":"@site/machine-learning/2026-06-09-gradient-descent-learning-rate-iterations.md","title":"Gradient Descent Learning Rate and Iterations","description":"In this short exercise, I explored how gradient descent reduces a cost function by repeatedly updating model parameters.{/truncate/}","date":"2026-06-09T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"Regression","permalink":"/machine-learning/tags/regression"},{"inline":true,"label":"Optimisation","permalink":"/machine-learning/tags/optimisation"},{"inline":true,"label":"Unit 8","permalink":"/machine-learning/tags/unit-8"}],"readingTime":2.41,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"}
1,"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"gradient-descent-learning-rate-iterations","title":"Gradient Descent Learning Rate and Iterations","authors":["rossbulat"],"tags":["Machine Learning","Regression","Optimisation","Unit 8"]},"unlisted":false,"prevItem":{"title":"Model Comparison, Explainability and Trust in Computer Vision","permalink":"/machine-learning/model-comparison-explainability-trust"},"nextItem":{"title":"Perceptron Notebook Reflections","permalink":"/machine-learning/perceptron-notebooks-reflection"}},"content":"In this short exercise, I explored how gradient descent reduces a cost function by repeatedly updating model parameters.{/*truncate*/}\\n\\n**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1yPzWbw-26cEq6dKDffKwK6gNVDckjwMP) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-06-09-gradient-descent-learning-rate-iterations.ipynb)\\n\\nThe tutorial uses a simple linear regression example where the input values are:\\n\\n```python\\nx = [1, 2, 3, 4, 5]\\ny = [5, 7, 9, 11, 13]\\n```\\n\\nThis data follows the relationship:\\n\\n```python\\ny = 2x + 3\\n```\\n\\nThe goal of gradient descent is to learn values for the slope `m` and intercept `b` that minimise the cost. In this case, the cost function is mean squared error, which measures how far the model\u2019s predictions are from the true values.\\n\\n## Experiment Setup\\n\\nI changed two values in the tutorial:\\n\\n- `iterations`: how many times gradient descent updates the parameters\\n- `learning_rate`: how large each update step is\\n\\nThe original tutorial uses:\\n\\n```python\\niterations = 100\\nlearning_rate = 0.08\\n```\\n\\nI tested several combinations and recorded the final cost.\\n\\n## Results\\n\\n| Experiment | Iterations | Learning Rate | Final Cost | Final `m` | Final `b` |\\n| ---------------------- | ---------: | ------------: | ---------: | ------------------------: | ------------------------: |\\n| Fewer iterations | 10 | 0.08 | 12.0468 | 1.6309 | 1.0383 |\\n| Default tutorial | 100 | 0.08 | 0.0041 | 2.0405 | 2.8536 |\\n| More iterations | 1000 | 0.08 | ~0.0000 | 2.0000 | 3.0000 |\\n| Small learning rate | 100 | 0.01 | 0.4804 | 2.4485 | 1.3809 |\\n| Moderate learning rate | 100 | 0.05 | 0.0321 | 2.1144 | 2.5870 |\\n| Too high learning rate | 100 | 0.10 | Diverged | Very large negative value | Very large negative value |\\n\\n## Observations\\n\\nIncreasing the number of iterations helped the model get closer to the correct line. After only 10 iterations, the cost had dropped from its initial value of `89` to about `12.05`, but the model was still far from the ideal values of `m = 2` and `b = 3`.\\n\\nAt 100 iterations and a learning rate of `0.08`, the cost fell to around `0.0041`. The learned parameters were close to the expected values, with `m \u2248 2.04` and `b \u2248 2.85`.\\n\\nWhen I extended the run to 1000 iterations while keeping the learning rate at `0.08`, the result was almost exact. The cost became effectively zero, and the model learned `m = 2` and `b = 3`.\\n\\nThe learning rate also had a major effect. A rate of `0.01` still reduced the cost, but convergence was slower. Increasing it to `0.05` performed better, while `0.08` gave the best result among the stable values I tested.\\n\\nIncreasing the learning rate too far caused the algorithm to diverge. At `learning_rate = 0.10`, the cost became extremely large instead of decreasing because the larger steps overshot the minimum.\\n\\n## Conclusion\\n\\nThe exercise made the relationship between iterations and learning rate clear. More iterations give the algorithm more chances to reduce the cost, but only when the learning rate is appropriate.\\n\\nA learning rate that is too small makes training slow, while a learning rate that is too large can make the model unstable and cause the cost to increase. In this example, `learning_rate = 0.08` with enough iterations gave the best balance between speed and stability."},{"id":"perceptron-notebooks-reflection","metadata":{"permalink":"/machine-learning/perceptron-notebooks-reflection","source":"@site/machine-learning/2026-06-02-perceptron-notebooks-reflection.md","title":"Perceptron Notebook Reflections","description":"I worked through three neural network notebooks: a simple perceptron, a perceptron trained for the AND operator, and a multilayer perceptron for a more complex non-linear problem.{/truncate/}","date":"2026-06-02T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"Neural Networks","permalink":"/machine-learning/tags/neural-networks"},{"inline":true,"label":"Unit 7","permalink":"/machine-learning/tags/unit-7"}],"readingTime":3.71,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"}
1,"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"perceptron-notebooks-reflection","title":"Perceptron Notebook Reflections","authors":["rossbulat"],"tags":["Machine Learning","Neural Networks","Unit 7"]},"unlisted":false,"prevItem":{"title":"Gradient Descent Learning Rate and Iterations","permalink":"/machine-learning/gradient-descent-learning-rate-iterations"},"nextItem":{"title":"A Hybrid Neural Network for Airbnb Demand","permalink":"/machine-learning/hybrid-airbnb-demand-network"}},"content":"I worked through three neural network notebooks: a simple perceptron, a perceptron trained for the AND operator, and a multilayer perceptron for a more complex non-linear problem.{/*truncate*/}\\n\\n## Simple perceptron\\n\\n**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/10TN8dXK93cr1zGtar1cAiZzPxhdgvaC5) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-06-02-simple-perceptron.ipynb)\\n\\nThe first notebook introduces the basic structure of a perceptron using NumPy arrays. One array holds the inputs, another holds the weights, and their dot product gives the weighted sum.\\n\\nThe main process is:\\n\\n1. Define input values.\\n2. Define weights.\\n3. Multiply inputs by weights using a dot product.\\n4. Pass the result into a step function.\\n5. Return either 1 or 0 depen
1ding on whether the weighted sum reaches the threshold.\\n\\nAt its simplest, a perceptron converts numeric inputs into a binary decision. In the notebook, adjusting the weights changes the final output, making their effect on the model\'s behaviour easy to see.\\n\\nThis simple example uses only one input vector. That makes the calculation easy to follow, but also highlights a limitation: a single perceptron calculation is only one small part of training a model for a real use case.\\n\\n## Perceptron AND operator\\n\\n**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1lzjCDocXcHt9TpaHw_xGRYLuuLonheLz) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-06-02-and-perceptron.ipynb)\\n\\nThe second notebook builds on this idea by using the full truth table for the AND operator:\\n\\n| Input 1 | Input 2 | Output |\\n|---:|---:|---:|\\n| 0 | 0 | 0 |\\n| 0 | 1 | 0 |\\n| 1 | 0 | 0 |\\n| 1 | 1 | 1 |\\n\\nHere, the input rows form a matrix, the targets are held in an array, and the weights start at zero. The notebook then defines a training loop. For each row, the perceptron makes a prediction, compares it with the expected output, calculates the error, and updates the weights when the prediction is wrong.\\n\\nThis is closer to a machine learning task because the model is not simply given fixed weights. Instead, it adjusts them over repeated passes until the total error reaches zero. The final weights then correctly classify all four AND cases.\\n\\nAND is linearly separable, so a single perceptron can solve it: one straight decision boundary separates the positive case `(1, 1)` from the other three cases. This dataset is useful for learning, but it is also very small and artificial. It does not include noise, ambiguity, missing values, or overlapping classes.\\n\\n## Multilayer perceptron\\n\\n**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1ad63dzs-ae6_FGoUpmQCrd6s9GBo4evF) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-06-02-multilayer-perceptron.ipynb)\\n\\nThe third notebook introduces a multilayer perceptron for a more complex target pattern. Its outputs match the XOR operator: `(0, 1)` and `(1, 0)` should return 1, while `(0, 0)` and `(1, 1)` should return 0.\\n\\nA single perceptron cannot solve XOR because the classes are not linearly separable. The notebook addresses this by adding a hidden layer between the input layer and the output layer. It also replaces the step function with the sigmoid activation function, which produces values between 0 and 1.\\n\\nThe training process is more involved:\\n\\n1. Send the inputs through the first set of weights into the hidden layer.\\n2. Apply the sigmoid function to the hidden-layer values.\\n3. Send the hidden-layer outputs through a second set of weights into the output layer.\\n4. Compare the predicted outputs with the target outputs.\\n5. Use the sigmoid derivative to calculate error signals.\\n6. Update both sets of weights through backpropagation.\\n7. Repeat this process for many epochs until the error becomes very small.\\n\\nThe hidden layer allows the network to learn an internal representation of the data rather than relying on a single straight boundary. This extra capacity comes at a cost: training is more computationally expensive and more sensitive to choices such as random weight initialisation, learning rate, number of epochs, and activation function.\\n\\n## Learning reflection\\n\\nA simple weighted sum demonstrates the mechanics of a perceptron, but it does not learn. A single perceptron can learn the AND dataset because the classes are linearly separable, whereas the XOR-style dataset needs a multilayer network because one linear boundary cannot separate the pattern.\\n\\nThe contrast also highlights why results from simple datasets do not transfer neatly to real-world problems, where data may contain noise, missing values, non-linear relationships, class imbalance, and ambiguous boundaries. As the dataset becomes more complex, the model may need more capacity, more training time, and more careful tuning."},{"id":"hybrid-airbnb-demand-network","metadata":{"permalink":"/machine-learning/hybrid-airbnb-demand-network","source":"@site/machine-learning/2026-05-28-hybrid-airbnb-demand-network.md","title":"A Hybrid Neural Network for Airbnb Demand","de
1scription":"Google Colab notebook Download .ipynb","date":"2026-05-28T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"Deep Learning","permalink":"/machine-learning/tags/deep-learning"},{"inline":true,"label":"Hybrid Models","permalink":"/machine-learning/tags/hybrid-models"},{"inline":true,"label":"Airbnb","permalink":"/machine-learning/tags/airbnb"},{"inline":true,"label":"Unit 6","permalink":"/machine-learning/tags/unit-6"}],"readingTime":3.81,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"},"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"hybrid-airbnb-demand-network","title":"A Hybrid Neural Network for Airbnb Demand","authors":["rossbulat"],"tags":["Machine Learning","Deep Learning","Hybrid Models","Airbnb","Unit 6"],"date":"2026-05-28T00:00:00.000Z"},"unlisted":false,"prevItem":{"title":"Perceptron Notebook Reflections","permalink":"/machine-learning/perceptron-notebooks-reflection"},"nextItem":{"title":"K-Means Clustering Notebook Methodology","permalink":"/machine-learning/k-means-clustering-notebook-methodology"}},"content":"**Google Colab notebook:** [Open the hybrid Airbnb demand notebook](https://colab.research.google.com/drive/1z1eTTGYFEgSsF1v2cRPx-uTsKunomePQ) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-05-28-hybrid-airbnb-demand-network.ipynb)\\n\\nI wanted to find out whether combining text, location context and ordinary listing details could predict Airbnb demand more effectively than structured data alone. I tested this using the Airbnb NYC 2019 dataset, with 48,884 listings remaining after invalid rows were removed. For this experiment, I defined \u201chigh demand\u201d as at least 1.58 reviews per month, the upper quartile of the dataset. This is a proxy for demand rather than a measure of confirmed bookings or occupancy, so the results should be interpreted as predictions of review activity.{/*truncate*/}\\n\\n## How the hybrid model works\\n\\nI used a leakage-aware stacking design with a held-out test set of 9,777 listings. I then split the remaining data into 19,553 rows for training two auxiliary models and a separate 19,554 rows for training the final classifier. This separation ensures that the final network does not learn from auxiliary predictions made on data those models have already seen.\\n\\nThe first branch uses a listing\u2019s name, host name and neighbourhood text to estimate the probability of positive sentiment. An embedding layer and LSTM learn to reproduce VADER\u2019s positive-versus-non-positive sentiment labels. This captures patterns in the listing metadata rather than sentiment from guest reviews. The second branch extracts a 32-by-32 greyscale map patch around each listing\u2019s coordinates, which a small convolutional network converts into a spatial demand probability.\\n\\nThe final classifier combines these two probabilities with log price, minimum nights, availability, latitude and longitude, host listing count, neighbourhood group and room type. After standardisation and one-hot encoding, the data passes through a neural network with 64-unit and 32-unit hidden layers. I used the same architecture, data split and random seed for the baseline, leaving out only the two auxiliary probabilities so that the comparison remained fair.\\n\\n## What the results show\\n\\n<img src=\\"/img/2026-05-28-hybrid-airbnb-demand-network/model-comparison.png\\" alt=\\"Notebook chart comparing held-out accuracy and ROC-AUC for the structured baseline and hybrid model\\" width=\\"700\\" style={{ display: \'block\', margin: \'2rem auto\' }} />\\n\\nAdding the text and map branches did not produce a clear overall improvement. The structured baseline achieved **0.7922 accuracy** and **0.8387 ROC-AUC**, while the latest hybrid evaluation reached **0.7930 accuracy** and **0.8359 ROC-AUC**. The hybrid gained less than one-tenth of a percentage point in accuracy but scored slightly lower on ROC-AUC. The map CNN al
1so produced probabilities clustered around 0.5, suggesting that the static map patches added little beyond the latitude, longitude and neighbourhood information already included in the structured data.\\n\\n<img src=\\"/img/2026-05-28-hybrid-airbnb-demand-network/hybrid-class-metrics.png\\" alt=\\"Chart comparing hybrid-model precision, recall and F1-score for standard-demand and high-demand listings\\" width=\\"600\\" style={{ display: \'block\', margin: \'2rem auto\' }} />\\n\\nAt the default threshold of 0.5, the hybrid model\u2019s precision, recall and F1-score were **0.64**, **0.41** and **0.50** for high-demand listings, compared with **0.82**, **0.92** and **0.87** for standard-demand listings. Its overall accuracy therefore hides much weaker performance on the minority class. If the model were being used to find as many high-demand listings as possible, I would consider a lower threshold, class-specific error costs and precision-recall analysis rather than solely relying on accuracy.\\n\\n<img src=\\"/img/2026-05-28-hybrid-airbnb-demand-network/score-band-demand-rate.png\\" alt=\\"Notebook chart showing observed high-demand rates rising across predicted probability bands from 5.5 percent to 85.3 percent\\" width=\\"700\\" style={{ display: \'block\', margin: \'2rem auto\' }} />\\n\\nEven so, the probabilities still provide a useful ranking. The observed high-demand rate rises from **5.5%** in the 0\u201320% prediction band to **85.3%** in the 80\u2013100% band. The top band contains only 34 listings, however, so its rate is based on a small sample, and this chart alone is not enough to show that the probabilities are well calibrated.\\n\\n## Takeaway\\n\\nThe hybrid architecture worked as intended, but the experiment did not justify its extra complexity. The sentiment branch learns from a proxy label, while the map branch overlaps with location features already available to the structured model. In this case, I would choose the structured model because its accuracy is effectively the same, its ROC-AUC is slightly stronger and it is easier to maintain. A more promising follow-up would be to test genuinely complementary inputs such as guest-review text, amenities, listing photographs or travel-time features."},{"id":"k-means-clustering-notebook-methodology","metadata":{"permalink":"/machine-learning/k-means-clustering-notebook-methodology","source":"@site/machine-learning/2026-05-28-k-means-clustering-methodology.md","title":"K-Means Clustering Notebook Methodology","description":"Google Colab notebook Download .ipynb","date":"2026-05-28T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"MSc","permalink":"/machine-learning/tags/m-sc"},{"inline":true,"label":"Data Analysis","permalink":"/machine-learning/tags/data-analysis"},{"inline":true,"label":"Unit 6","permalink":"/machine-learning/tags/unit-6"}],"readingTime":3.93,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"},"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"k-means-clustering-notebook-methodology","title":"K-Means Clustering Notebook Methodology","authors":["rossbulat"],"tags":["Machine Learning","MSc","Data Analysis","Unit 6"]},"unlisted":false,"prevItem":{"title":"A Hybrid Neural Network for Airbnb Demand","permalink":"/machine-learning/hybrid-airbnb-demand-network"},"nextItem":{"title":"K-Means Clustering Animation Reflections","permalink":"/machine-learning/k-means-clustering-animation-reflections"}},"content":"**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1swFYq3pL2BC26T9bXYqKvMBSCymky8u-) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-05-28-k-means-clustering-methodology.ipynb)\\n\\nMy Colab notebook applies the same k-means clustering process to three datasets: Iris, Wine, and WeatherAUS. Here, I explain the methodology behind it.{/*truncate*/}\\n\\n## Methodology\\n\\nThe notebook uses a consistent unsupervised learning workflow. Each dataset is loaded from CSV, inspected briefly, and then converted into a numeric feature matrix suitable for k-means. Labels and outcome columns are removed before clustering because k-means should group records by feature similarity, not by being given the answer in advance.\\n\\nStandardisation is the most important preprocessing step. k-means uses Euclidean distance, so variables with larger numeric ranges can dominate the result if they are left unscaled. The notebook therefore uses median imputation for missing numeric values and `StandardScaler` to put all selected features onto a comparable scale.\\n\\nFor each dataset, I explore different values of `K` using:\\n\\n- SSE / inertia, shown through elbow plots.\\n- Silhouette scores, used to assess how separated the cluster
1s are.\\n- PCA scatter plots, used to visualise high-dimensional clusters in two dimensions.\\n\\nThe model uses `k-means++` initialisation, repeated initialisation with `n_init=12`, a fixed random seed, and a maximum of 300 iterations. Together, these settings improve stability and reproducibility across repeated runs of the iterative algorithm.\\n\\n## Iris and Wine\\n\\nFor the Iris dataset, the notebook removes the `species` label and clusters the four numeric flower measurements: sepal length, sepal width, petal length, and petal width. The model uses `K = 3`, which matches the three known species. The clusters are then compared with the true species labels using crosstabs, Adjusted Rand Index, Normalised Mutual Information, and best mapped cluster-label accuracy.\\n\\nK-means separates `setosa` very clearly, but there is some overlap between `versicolor` and `virginica`. This was a useful reminder that the algorithm can identify structure, but it does not understand botanical categories. It only uses distance between numeric measurements.\\n\\nFor the Wine dataset, the notebook removes the `Wine` class label and clusters thirteen chemical measurements. Again, `K = 3` is used so the unsupervised clusters can be compared with the three known wine classes. The Wine result is stronger than the Iris result, with a best mapped cluster-label accuracy of 0.966, suggesting that the chemical variables contain a clearer grouping structure for k-means.\\n\\n## WeatherAUS\\n\\nThe WeatherAUS dataset is more challenging. It contains numeric weather measurements, categorical fields, dates, locations, and rain labels. For the k-means analysis, the notebook removes `RainTomorrow` and the categorical `RainToday` field, then uses twelve numeric weather variables including temperature, rainfall, wind speed, humidity, pressure, and time-of-day temperature readings.\\n\\nThe notebook tests `K = 2` to `K = 6`, then visualises each result using PCA. A `K = 3` model is inspected in more detail. The cluster summaries suggest broad weather profiles, such as warmer conditions, wetter and windier conditions, and cooler higher-pressure conditions. The optional comparison with `RainTomorrow` is useful, but it should not be treated as a supervised prediction task. K-means has not been trained to predict rain; it has only grouped records by similarity.\\n\\n## Learning reflection\\n\\nComparing the three datasets helped me see where this method is applicable and where its limitations become more apparent. Iris and Wine are clean, compact datasets with known labels, so they are useful for testing whether unsupervised clusters align with expected classes. The WeatherAUS dataset is more realistic because it includes missing values, mixed data types, and a much larger number of observations. This makes preprocessing decisions more important and also makes the clusters harder to explain.\\n\\nOne limitation is that clustering can create categories that look authoritative even when the groupings are only a mathematical result of the algorithm. If this method were used on people, such as customers, patients, students, or employees, the selected features, value of `K`, missing-data treatment, and interpretation of clusters would need to be documented carefully. Poorly explained clusters could lead to unfair segmentation, bi
1ased decisions, or misleading claims about groups.\\n\\nI also organised the notebook with readability in mind. Reusable helper functions handle scaling, metric plots, PCA visualisations, and label comparisons, making the analysis easier to review, rerun, or extend in a virtual environment. Fixed random seeds, explicit feature lists, and clear separation between preprocessing, modelling, visualisation, and interpretation support reproducibility.\\n\\n## Summary\\n\\nThe notebook demonstrates that k-means is a practical method for exploratory clustering when the data is numeric, scaled, and suitable for distance-based grouping. It works especially well when clusters are compact and well separated, as seen in parts of the Iris and Wine analysis. However, the WeatherAUS task shows that real-world data requires more careful preprocessing and more cautious interpretation."},{"id":"k-means-clustering-animation-reflections","metadata":{"permalink":"/machine-learning/k-means-clustering-animation-reflections","source":"@site/machine-learning/2026-05-21-k-means-clustering.md","title":"K-Means Clustering Animation Reflections","description":"I watched two animations of the k-means clustering algorithm to see how it repeatedly assigns data points to clusters and updates their centroids.{/truncate/}","date":"2026-05-21T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"MSc","permalink":"/machine-learning/tags/m-sc"},{"inline":true,"label":"Data Analysis","permalink":"/machine-learning/tags/data-analysis"},{"inline":true,"label":"Unit 5","permalink":"/machine-learning/tags/unit-5"}],"readingTime":1.53,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"},"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"k-means-clustering-animation-reflections","title":"K-Means Clustering Animation Reflections","authors":["rossbulat"],"tags":["Machine Learning","MSc","Data Analysis","Unit 5"]},"unlisted":false,"prevItem":{"title":"K-Means Clustering Notebook Methodology","permalink":"/machine-learning/k-means-clustering-notebook-methodology"},"nextItem":{"title":"Calculating the Jaccard Coefficient","permalink":"/machine-learning/calculating-jaccard-coefficient"}},"content":"I watched two animations of the k-means clustering algorithm to see how it repeatedly assigns data points to clusters and updates their centroids.{/*truncate*/}\\n\\nThe animations are sourced from:\\n\\n- [Animation 1: K-Means Clustering](http://shabal.in/visuals/kmeans/2.html)\\n- [Animation 2: K-Means Clustering](https://www.naftaliharris.com/blog/visualizing-k-means-clustering/)\\n\\n\\n\\n*Figure 1. Selected stages from the first animation: initial centroid placement, an intermediate assignment and update step, and the final clusters.*\\n\\n## Reflection\\n\\nBoth animations show the iterative process behind k-means:\\n\\n- choose the number of clusters\\n- place initial centroids\\n- assign each point to its nearest centroid\\n- update each centroid to the mean of its assigned points\\n- repeat until the clusters stabilise\\n\\nThe first animation shows how the starting position of centroids can strongly affect the early clustering process. Even if the initial placement is poor, the algorithm gradually improves the grouping by repeatedly reassigning points and moving centroids.\\n\\nThe \u201cI\u2019ll cho
1ose\u201d and \u201cUniform Points\u201d options in the second animation show that k-means will still create clusters even when the data may not contain meaningful natural groups. This happens because the algorithm optimises distance and has no understanding of real-world meaning or context. It works best when clusters are relatively compact and well separated, but it can be misleading with uniform, irregular, overlapping, or high-dimensional datasets.\\n\\nFrom an ethical perspective, if clustering is used to segment people, such as customers, students, patients, or job applicants, the chosen value of k, dataset features, and assumptions behind the model need to be justified. Otherwise, the algorithm may create artificial categories and lead to decisions that are biased, unfair, or difficult to explain."},{"id":"calculating-jaccard-coefficient","metadata":{"permalink":"/machine-learning/calculating-jaccard-coefficient","source":"@site/machine-learning/2026-05-20-jaccard-coefficient.md","title":"Calculating the Jaccard Coefficient","description":"Introduction","date":"2026-05-20T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"Introduction","permalink":"/machine-learning/tags/introduction"},{"inline":true,"label":"Unit 5","permalink":"/machine-learning/tags/unit-5"}],"readingTime":5.47,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"},"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"calculating-jaccard-coefficient","title":"Calculating the Jaccard Coefficient","authors":["rossbulat"],"tags":["Machine Learning","Introduction","Unit 5"]},"unlisted":false,"prevItem":{"title":"K-Means Clustering Animation Reflections","permalink":"/machine-learning/k-means-clustering-animation-reflections"},"nextItem":{"title":"Global Population and GDP - Correlation and Regression","permalink":"/machine-learning/global-population-gdp-correlation-regression"}},"content":"## Introduction\\n\\nI used a small table of pathological test results to calculate the **Jaccard distance** and corresponding **Jaccard similarity coefficient** for three pairs of individuals.{/*truncate*/} The three individuals are **Jack**, **Mary** and **Jim**, and the required comparisons are:\\n\\n- Jack and Mary\\n- Jack and Jim\\n- Jim and Mary\\n\\n## What is the Jaccard coefficient?\\n\\nThe **Jaccard similarity coefficient** is a similarity measure for binary attributes. It compares two records by computing the ratio of matches where both records have a value of 1 to the number of attributes where at least one record has a value of 1.\\n\\nThe formula is:\\n\\n**J(A, B) = f\u2081\u2081 / (f\u2080\u2081 + f\u2081\u2080 + f\u2081\u2081)**\\n\\nThe corresponding **Jaccard distance**, or dissimilarity, is the proportion of mismatches:\\n\\n**d\u2c7c(A, B) = (f\u2080\u2081 + f\u2081\u2080) / (f\u2080\u2081 + f\u2081\u2080 + f\u2081\u2081) = 1 - J(A, B)**\\n\\nWhere:\\n\\n- **f\u2081\u2081** = number of attributes where both records have value 1\\n- **f\u2080\u2081** = number of attributes where record A has 0 and record B has 1\\n- **f\u2081\u2080** = number of attributes where record A has 1 and record B has 0\\n- Matches where both have 0 are ignored in the denominator.\\n\\nFor this activity, each person is treated as a set of **attribute-value pairs**, excluding the `Name` field. For example, Jack\u2019s record is represented as:\\n\\n- Gender = M\\n- Fever = Y\\n- Cough = N\\n- Test-1 = P\\n- Test-2 = N\\n- Test-3 = N\\n- Test-4 = A\\n\\n## Binary Conversion\\n\\n**Asymmetric variables** are converted to binary:\\n- Y and P \u2192 **1**\\n- N and A \u2192 **0**\\n\\n**Gender is NOT converted** because it is a **symmetric variable** (male and female have equal weight). It is retained only to check that the records are comparable.\\n\\nAfter binary conversion, each person has **6 binary attributes** (excluding Name and the unconverted Gender). For example, Jack\'s record becomes:\\n\\n- Gender = M (not converted)\\n- Fever = Y \u2192 1\\n- Cough = N \u2192 0\\n- Test-1 = P \u2192 1\\n- Test-2 = N \u2192 0\\n- Test-3 = N \u2192 0\\n- Test-4 = A \u2192 0\\n\\n## Source Data\\n\\n| Name | Gender | Fever | Cough | Test-1 | Test-2 | Test-3 | Test-4 |\\n|---|---|---|---|---|---|---|---|\\n| Jack | M | Y | N | P | N | N | A |\\n| Mary | F | Y | N | P | A | P | N |\\n| Jim | M | Y | P | N | N | N | A |\\n\\n## Calculation Method\\n\\
1nFor each pair of individuals, I converted asymmetric variables to binary (Y & P = 1; N & A = 0), excluded Gender from binary conversion, and compared the 6 binary attributes.\\n\\nFor each pair, I calculated:\\n\\n1. **f\u2081\u2081**: Count of attributes where both individuals have value 1\\n2. **f\u2080\u2081 + f\u2081\u2080**: Count of attributes where one has 1 and the other has 0\\n3. The Jaccard distance using: **d\u2c7c = (f\u2080\u2081 + f\u2081\u2080) / (f\u2080\u2081 + f\u2081\u2080 + f\u2081\u2081)**\\n4. The Jaccard similarity coefficient using: **J = f\u2081\u2081 / (f\u2080\u2081 + f\u2081\u2080 + f\u2081\u2081)**\\n\\n---\\n\\n## Pair 1: Jack and Mary\\n\\n### Original Data\\n\\n**Jack:** Gender=M, Fever=Y, Cough=N, Test-1=P, Test-2=N, Test-3=N, Test-4=A \\n**Mary:** Gender=F, Fever=Y, Cough=N, Test-1=P, Test-2=A, Test-3=P, Test-4=N\\n\\n### Binary Conversion\\n\\n**Jack (binary):** Gender=M (not converted), 1, 0, 1, 0, 0, 0 \\n**Mary (binary):** Gender=F (not converted), 1, 0, 1, 0, 1, 0\\n\\n### Attribute Comparison\\n\\nComparing the 6 binary attributes (skipping Gender):\\n\\n| Position | Attribute | Jack | Mary | Match Type |\\n|---|---|---:|---:|---|\\n| 1 | Fever | 1 | 1 | f\u2081\u2081 (both 1) |\\n| 2 | Cough | 0 | 0 | Ignored |\\n| 3 | Test-1 | 1 | 1 | f\u2081\u2081 (both 1) |\\n| 4 | Test-2 | 0 | 0 | Ignored |\\n| 5 | Test-3 | 0 | 1 | f\u2080\u2081 (mismatch) |\\n| 6 | Test-4 | 0 | 0 | Ignored |\\n\\n### Calculation\\n\\n- **f\u2081\u2081** = 2 (Fever, Test-1)\\n- **f\u2080\u2081 + f\u2081\u2080** = 1 (Test-3)\\n\\nJaccard distance:\\n\\n**d\u2c7c(Jack, Mary) = (1) / (1 + 2) = 1/3 = 0.333**\\n\\n### Result\\n\\nThe Jaccard distance for **Jack and Mary** is:\\n\\n**0.33**\\n\\n---\\n\\n## Pair 2: Jack and Jim\\n\\n### Original Data\\n\\n**Jack:** Gender=M, Fever=Y, Cough=N, Test-1=P, Test-2=N, Test-3=N, Test-4=A \\n**Jim:** Gender=M, Fever=Y, Cough=P, Test-1=N, Test-2=N, Test-3=N, Test-4=A\\n\\n### Binary Conversion\\n\\n**Jack (binary):** Gender=M (not converted), 1, 0, 1, 0, 0, 0 \\n**Jim (binary):** Gender=M (not converted), 1, 1, 0, 0, 0, 0\\n\\n### Attribute Comparison\\n\\nComparing the 6 binary attributes (skipping Gender):\\n\\n| Position | Attribute | Jack | Jim | Match Type |\\n|---|---|---:|---:|---|\\n| 1 | Fever | 1 | 1 | f\u2081\u2081 (both 1) |\\n| 2 | Cough | 0 | 1 | f\u2080\u2081 (mismatch) |\\n| 3 | Test-1 | 1 | 0 | f\u2081\u2080 (mismatch) |\\n| 4 | Test-2 | 0 | 0 | Ignored |\\n| 5 | Test-3 | 0 | 0 | Ignored |\\n| 6 | Test-4 | 0 | 0 | Ignored |\\n\\n### Calculation\\n\\n- **f\u2081\u2081** = 1 (Fever)\\n- **f\u2080\u2081 + f\u2081\u2080** = 2 (Cough, Test-1)\\n\\nJaccard distance:\\n\\n**d\u2c7c(Jack, Jim) = (2) / (2 + 1) = 2/3 = 0.667**\\n\\n### Result\\n\\nThe Jaccard distance for **Jack and Jim** is:\\n\\n**0.67**\\n\\n---\\n\\n## Pair 3: Jim and Mary\\n\\n### Original Data\\n\\n**Jim:** Gender=M, Fever=Y, Cough=P, Test-1=N, Test-2=N, Test-3=N, Test-4=A \\n**Mary:** Gender=F, Fever=Y, Cough=N, Test-1=P, Test-2=A, Test-3=P, Test-4=N\\n\\n### Binary Conversion\\n\\n**Jim (binary):** Gender=M (not converted), 1, 1, 0, 0, 0, 0 \\n**Mary (binary):** Gender=F (not converted), 1, 0, 1, 0, 1, 0\\n\\n### Attribute Comparison\\n\\nComparing the 6 binary attributes (skipping Gender):\\n\\n| Position | Attribute | Jim | Mary | Match Type |\\n|---|---|---:|---:|---|\\n| 1 | Fever | 1 | 1 | f\u2081\u2081 (both 1) |\\n| 2 | Cough | 1 | 0 | f\u2081\u2080 (mismatch) |\\n| 3 | Test-1 | 0 | 1 | f\u2080\u2081 (mismatch) |\\n| 4 | Test-2 | 0 | 0 | Ignored |\\n| 5 | Test-3 | 0 | 1 | f\u2080\u2081 (mismatch) |\\n| 6 | Test-4 | 0 | 0 | Ignored |\\n\\n### Calculation\\n\\n- **f\u2081\u2081** = 1 (Fever)\\n- **f\u2080\u2081 + f\u2081\u2080** = 3 (Cough, Test-1, Test-3)\\n\\nJaccard distance:\\n\\n**d\u2c7c(Jim, Mary) = (3) / (3 + 1) = 3/4 = 0.75**\\n\\n### Result\\n\\nThe Jaccard distance for **Jim and Mary** is:\\n\\n**0.75**\\n\\n---\\n\\n## Summary of Results\\n\\n### Jaccard Distance\\n\\n| Pair | f\u2081\u2081 | f\u2080\u2081 + f\u2081\u2080 | Jaccard Distance |\\n|---|---:|---:|---:|\\n| Jack and Mary | 2 | 1 | 0.33 |\\n| Jack and Jim | 1 | 2 | 0.67 |\\n| Jim and Mary | 1 | 3 | 0.75 |\\n\\n### Jaccard Similarity Coefficient\\n\\nThe Jaccard similarity coefficient is the complement of the Jaccard distance: **J = 1 - d\u2c7c**.\\n\\n| Pair | Calculation | Jaccard Similarity Coefficient |\\n|---|---:|---:|\\n| Jack and Mary | 2 / (2 + 1) | 0.67 |\\n| Jack and Jim | 1 / (1 + 2) | 0.33 |\\n| Jim and Mary | 1 / (1 + 3) | 0.25 |\\n\\n## Interpretation\\n\\n**Jack and Mary** have the highest similarity, with a Jaccard similarity coefficient of **0.67** and a Jaccard distance of **0.33**. They have the fewest mismatches among attributes where at least one individual has a value of 1.\\n\\n**Jack and Jim** have moderate similarity, with a Jaccard similarity coefficient of **0.33** and a Jaccard distance of **0.67**. They differ on the Cough and Test-1 attributes while sharing the Fever characteristic.\\n\\n**Jim and Mary** have the lowest similarity, with a Jaccard similarity coefficient of **0.25** and a Jaccard distance of **0.75**. They have the most mismatches among 1-valued attributes: Cough, Test-1, and Test-3.\\n\\nHigher Jaccard similarity coefficients indicate greater similarity, while lower Jaccard distances indicate greater similarity."},{"id":"global-population-gdp-correlation-regression","metadata":{"permalink":"/machine-learning/global-population-gdp-correlation-regression","source":"@site/machine-learning/2026-05-19-global-population-gdp-correlation.md","title":"Global Population and GDP - Correlation and Regression","description":"Google Colab notebook Download .ipynb","date":"2026-05-19T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"MSc","permalink":"/machine-learning/tags/m-sc"},{"inline":true,"label":"Data Analysis","permalink":"/machine-learning/tags/data-analysis"},{"inline":true,"label":"Unit 4","permalink":"/machine-learning/tags/unit-4"}],"readingTime":7.15,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"}
1,"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"global-population-gdp-correlation-regression","title":"Global Population and GDP - Correlation and Regression","authors":["rossbulat"],"tags":["Machine Learning","MSc","Data Analysis","Unit 4"]},"unlisted":false,"prevItem":{"title":"Calculating the Jaccard Coefficient","permalink":"/machine-learning/calculating-jaccard-coefficient"},"nextItem":{"title":"Collaborative Discussion 1 Reflection","permalink":"/machine-learning/collab-1-reflection"}},"content":"**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1wuWIf0Gqoqx6v0BC2DRZ1QjlnQ7zHPv5) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-05-19-global-population-gdp-correlation.ipynb)\\n\\nThis portfolio activity investigates the relationship between a country or entity\'s **mean population** and its **mean total GDP** across the years **2001-2020**. The analysis uses two World Bank datasets, `global_gdp.csv` and `global_population.csv`, and applies descriptive analysis, Pearson correlation, log-scale visualisation, and simple linear regression.{/*truncate*/}\\n\\n## Data and preprocessing\\n\\nThe two CSV files are loaded directly from URLs, so the notebook runs end-to-end in Google Colab without manual uploads:\\n\\n- `global_gdp.csv`\\n- `global_population.csv`\\n\\nThe raw files contain 266 GDP rows and 272 population rows. A shared helper function, `prepare_country_year_values`, handles preprocessing for both datasets. It:\\n\\n- Selects the country/entity name, country code, and year columns for 2001-2020.\\n- Converts yearly values to numeric values.\\n- Interpolates across year columns to fill isolated missing values.\\n- Drops rows where all selected year values are still missing.\\n- Renames columns into a consistent format for merging.\\n\\nThe cleaned GDP and population tables are merged by country code, producing **257 rows** and **42 columns**. The analysis then calculates:\\n\\n- **Mean population**, 2001-2020.\\n- **Mean GDP**, 2001-2020.\\n- Mean population in **millions**.\\n- Mean GDP in **billions** and **trillions of US dollars**.\\n\\nRows are kept only where both mean population and mean GDP are positive, which is also required for the later log-scale plot. The final analysis contains **257 countries/entities**.\\n\\n### Scope of the rows\\n\\nThe notebook analyses **countries/entities**. World Bank aggregate rows such as `World`, `High income`, `OECD members`, and regional or income-group totals remain in the dataset. This is notable when interpreting the results because those aggregate entities are much larger than individual countries and are not statistically independent of the countries they contain.\\n\\n## Descriptive analysis\\n\\nBoth variables are highly skewed. The median entity has a mean population of about **9.43 million** and mean GDP of about **US$39.46 billion**, while the largest entity, `World`, has a mean population of about **6.97 billion** and mean GDP of about **US$66.70 trillion**.\\n\\nThe largest mean-population observations are aggregate entities:\\n\\n| Entity | Mean population | Mean GDP |\\n|---|---:|---:|\\n| World | 6,971.25 million | US$66.70 trillion |\\n| IDA & IBRD total | 5,842.84 million | US$22.86 trillion |\\n| Low & middle income | 5,790.14 million | US$21.93 trillion |\\n| Middle income | 5,266.19 million | US$21.55 trillion |\\n| Early-demographic dividend | 2,927.29 million | US$9.85 trillion |\\n\\nThe highest mean-GDP observations are also mostly aggregate entities, including `World`, `High income`, `OECD members`, and `Post-demographic dividend`.\\n\\n## Task A: Correlation and visual pattern\\n\\nThe Pearson correlation coefficient was computed between mean population, measured in millions, and mean total GDP, measured in trillions of US dollars:\\n\\n- **Pearson r = 0.7213**\\n- **p-value = 1.459e-42**\\n\\nThis is a **strong positive linear relationship** in the analysed dataset. Entities with larger populations generally have larger total GDP.\\n\\nThis pattern makes sense because total GDP measures aggregate output. A larger population can mean a larger workforce, larger domestic market, and greater total production. However, the relationship is not perfect: population size alone does not determine how productive an economy is or how much output is produced per person.\\n\\n### Linear-scale plot\\n\\nThe linear-scale scatter plot shows a visible upward pattern between mean population and mean total GDP. The largest aggregate entities sit toward the upper-right of the plot, while many smaller countries/entities cluster near the lower-left.\\n\\nThere is also substantial spread around the trend. Some smaller-population economies have relatively high GDP, and some larger-population entities have lower GDP than the trend would predict. This variation suggests that population is useful but incomplete as a single explanatory variable.\\n\\n### Log-scale visual check\\n\\nBecause both population and GDP are highly skewed, the notebook also plots both axes on a log scale. The log transformation makes smaller and medium-sized entries easier to compare.\\n\\nAfter applying the log transform:\\n\\n- **Pearson r = 0.8818**\\n- **p-value = 3.15e-85**\\n\\nThe positive relationship remains and becomes visually clearer on the log-log plot. The notebook also identifies a small top-left cluster: countries/entities with smaller populations but comparatively high GDP. In the displayed examples, **Switzerland** appears in this group, with a mean population of about **7.90 million** and mean GDP of about **US$0.582 trillion**.\\n\\nTaken together, the plots show that population is related to total GDP, but GDP also depen
1ds on productivity, industry structure, trade, institutions, education, infrastructure, technology, natural resources, and historical development.\\n\\n## Task B: Linear regression\\n\\nA simple linear regression model was fitted with mean population as the independent variable and mean total GDP as the dependent variable:\\n\\n**Mean GDP = beta_0 + beta_1 x Mean population**\\n\\nFor readability:\\n\\n- Mean population is measured in **millions of people**.\\n- Mean GDP is measured in **trillions of US dollars**.\\n\\nThis means the slope represents the expected change in mean GDP, in trillions of US dollars, for each additional one million people.\\n\\n### Results\\n\\n| Metric | Value |\\n|---|---:|\\n| Intercept | US$0.4497 trillion |\\n| Slope, trillions per 1 million people | US$0.006036 trillion |\\n| Slope, billions per 1 million people | US$6.036 billion |\\n| R-squared | 0.5203 |\\n| RMSE | US$4.958 trillion |\\n| Number of observations | 257 |\\n\\nThe fitted equation is:\\n\\n**Mean GDP in US$ trillions = 0.4497 + 0.006036 x mean population in millions**\\n\\n### Interpretation\\n\\n- The fitted **slope is positive**.\\n- Each additional 1 million people is associated with an estimated **US$6.036 billion** increase in mean total GDP.\\n- The **R-squared value of 0.5203** means the model explains about **52.03%** of the variation in mean total GDP across the analysed countries/entities.\\n- The **RMSE of US$4.958 trillion** is large, which shows that the model\'s predictions can still be far from observed GDP values.\\n- The intercept is merely a mathematical reference point for the regression line. A country or entity with zero population is not a meaningful real-world case.\\n\\nThe model is useful, but its limits are clear. Population explains a substantial share of the variation in total GDP in this dataset, but it is still a one-variable model and should not be treated as a complete explanation of economic output.\\n\\n### Example predictions\\n\\nThe notebook demonstrates predictions for several population sizes:\\n\\n| Mean population | Predicted mean GDP |\\n|---:|---:|\\n| 10 million | US$0.510 trillion |\\n| 50 million | US$0.751 trillion |\\n| 100 million | US$1.053 trillion |\\n| 500 million | US$3.468 trillion |\\n\\nThese values illustrate the fitted regression line; they are not forecasts. They use population as the only input, so they ignore productivity and other economic differences between countries/entities.\\n\\n## Reflections\\n\\n### Legal, social, ethical and professional issues\\n\\nThe issues raised by country-level and aggregate economic data extend beyond model accuracy:\\n\\n- **Risk of misinterpretation.** A positive relationship between population and total GDP should not be presented as \\"larger populations make countries richer\\". Total GDP is not the same as GDP per capita or living standards.\\n- **Aggregate rows.** The dataset includes World Bank aggregate entities such as `World`, income groups, and regions. These rows are useful for a broad descriptive exercise, but they overlap with country rows and are not independent observations.\\n- **Data provenance and quality.** World Bank data is compiled from national and institutional sources. Measurement quality can vary by country, year, and economic context.\\n- **Causal limits.** The regression is associational. It does not prove that population growth causes GDP growth, nor does it account for productivity, institutions, trade, education, infrastru
1cture, conflict, or resource endowments.\\n\\n### Applicability and challenges of the dataset\\n\\n- **Skewed distributions.** Population and total GDP are both heavily right-skewed, so log-scale visualisation is useful for revealing patterns among smaller and medium-sized entries.\\n- **Single predictor.** Population alone explains about half of the observed variation in total GDP, but the remaining variation is still large.\\n- **Unit interpretation.** The model uses population in millions and GDP in trillions for readability. The unit conversion does not change the relationship, but it matters for interpreting the slope correctly.\\n- **Entity definition.** Because aggregate entities are included, the results should be described as applying to \\"countries/entities\\" rather than strictly to \\"countries\\".\\n- **GDP versus GDP per capita.** Total GDP captures aggregate economic output. It should not be used as a direct measure of individual prosperity.\\n\\n## Summary\\n\\nAcross 257 countries/entities between 2001 and 2020, mean population and mean total GDP show a **strong positive correlation** (Pearson **r = 0.7213**, **p = 1.459e-42**). After log transformation, the relationship is even stronger (Pearson **r = 0.8818**, **p = 3.15e-85**).\\n\\nThe simple linear regression model estimates that each additional 1 million people is associated with about **US$6.036 billion** more mean total GDP, and population alone explains about **52.03%** of the variation in mean GDP. Overall, **population is meaningfully related to total GDP, but it is not a complete explanation of economic output or prosperity**."},{"id":"collab-1-reflection","metadata":{"permalink":"/machine-learning/collab-1-reflection","source":"@site/machine-learning/2026-05-14-collab-1-reflection.md","title":"Collaborative Discussion 1 Reflection","description":"The first collaborative discussion in the Machine Learning module focused on Industry 4.0, Industry 5.0, digital infrastructure and data-driven operational resilience. Here, I reflect on my initial post, two peer responses and the final summary.{/ truncate /}","date":"2026-05-14T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"MSc","permalink":"/machine-learning/tags/m-sc"},{"inline":true,"label":"Reflection","permalink":"/machine-learning/tags/reflection"},{"inline":true,"label":"Unit 1","permalink":"/machine-learning/tags/unit-1"},{"inline":true,"label":"Unit 2","permalink":"/machine-learning/tags/unit-2"},{"inline":true,"label":"Unit 3","permalink":"/machine-learning/tags/unit-3"}],"readingTime":5.26,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"},"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"collab-1-reflection","title":"Collaborative Discussion 1 Reflection","authors":["rossbulat"],"tags":["Machine Learning","MSc","Reflection","Unit 1","Unit 2","Unit 3"]},"unlisted":false,"prevItem":{"title":"Global Population and GDP - Correlation and Regression","permalink":"/machine-learning/global-population-gdp-correlation-regression"},"nextItem":{"title":"Airbnb NYC Dataset - Demand Proxy Rationale","permalink":"/machine-learning/airbnb-nyc-demand-proxy-rationale"}},"content":"The first collaborative discussion in the Machine Learning module focused on Industry 4.0, Industry 5.0, digital infrastructure and data-driven operational resilience. Here, I reflect on my initial post, two peer responses and the final summary.{/* truncate */}\\n\\n## Initial Post Reflection\\n\\nIn my initial post, I went beyond treating Industry 4.0 only as a smart factory concept and connected it to software, cloud platforms, cyber-physical systems, automated security tooling and connected digital infrastructure.\\n\\nI think the CrowdStrike Falcon outage was the strongest part of the post. It gave the argument a concrete example and showed how a technical update failure can become a wider organisational and societal disruption. It also let me link Industry 5.0 to resilience, human oversight and stakeholder value, positioning it not as a replacement for Industry 4.0 but as a corrective perspective that asks whether automation is safe, accountable and sustainable at scale.\\n\\nThat said, the post tried to do too much at once. It introduced Industry 4.0, Industry 5.0, the CrowdStrike incident, analytics, machine learning, data quality, validation, release management and virtual professional teamwork in a short space. Although the result was coherent, some ideas were compressed rather than fully examined.\\n\\nThe link to machine learning and data quality was valid but underdeveloped: I did not show exactly how EDA or ML methods would help prevent or diagnose this kind of failure. I would also be more critical of the proposed safeguards. Staged rollouts, validation and rollback mechanisms sound persuasive, but I did not question why these measures fail in organisations that already know they are important.\\n\\n## Exchange with Abdullah\\n\\
1nAbdullah\'s response to my initial post strengthened the systemic-risk argument. He pointed out that failures in software systems are no longer isolated technical events but value-chain disruptions affecting sectors such as aviation, healthcare and finance. He also added a useful point about human-in-the-loop validation, which shifted the discussion from broad human-centric design to concrete validation practices before production release.\\n\\nI accepted that extension too readily. A stronger reply would have asked what \\"human-in-the-loop\\" actually means in fast-moving cybersecurity environments, where human review may reduce some risks but can also become a bottleneck, a symbolic control, or an overloaded approval stage when updates are released at high frequency.\\n\\nMy response to Abdullah\'s post on the Facebook, WhatsApp and Instagram outage was one of my stronger contributions because it was more concrete. I linked his platform-dependence example to the wider risks of Industry 4.0-style digital infrastructure and moved beyond agreement by focusing on prevention: change-management controls, staged rollout procedures, automated rollback mechanisms, failover pathways and incident communication. In doing so, I treated centralised digital services as critical infrastructure for communication, sales and customer engagement.\\n\\nI could have been sharper in comparing the Meta outage with the CrowdStrike incident. Both involved configuration or update-related failure, but the Meta outage was mainly about internal network configuration and platform availability, while CrowdStrike involved endpoint security software affecting customer machines across many external organisations. That distinction would have let me compare different forms of systemic risk. I should also have explained more precisely what questions EDA would answer about network logs, traffic anomalies and dependency maps, rather than just naming the method.\\n\\n## Peer Response Reflection: Ariel\'s Post\\n\\nAriel\'s post broadened the discussion by moving from digital platforms into automotive manufacturing, showing that Industry 4.0 risk is not limited to software companies.\\n\\nIn my response, I explored the tension between the benefits of automation and its resilience risks. I agreed that BMW\'s AI maintenance example showed the positive side of Industry 4.0, but argued that the same proactive logic should apply to cyber and operational resilience. I also used the Jaguar Land Rover cyber incident to show that smart factory performance depends on business continuity, supply-chain stability and recovery planning, not only automation and productivity.\\n\\nThe governance link was the part I found most effective. Using the NIST Cybersecurity Framework, I framed cyber resilience as an organisational issue involving governance, identification, protection, detection, response and recovery, which fits the Industry 5.0 emphasis on socio-technical systems.\\n\\nHowever, the response became solution-heavy. I listed measures such as supplier access controls, compartmentalised networks, recovery plans and anomaly monitoring without fully exploring their trade-offs. Stricter controls may slow production workflows, increase compliance overhead, or create tension between operational efficiency and cyber resilience. A more critical response would have acknowledged that Industry 5.0 is not just about adding safeguards, but about managing conflicts between speed, cost, productivity, worker impact and resilience.\\n\\n## Summary Post Reflection\\n\\nThe summary post was more focused than the initial post and showed how my thinking had developed. By then, my central argument was clearer: Industry 5.0 acts as a corrective layer that asks whether automation is resilient, human-centred and accountable.\\n\\nI also made a clearer connection to machine learning practice through EDA, log analysis and MLOps. This gave more attention to data quality, monitoring and structured evidence from operational systems, developing the ML connection that had been too brief in my initial post.\\n\\nThe summary still leaned toward an oversimplified governance conclusion. It argued that staged deployment, rollback mechanisms, failover pathways, incident ownership, EDA and MLOps can improve resilience, but did not fully address why organisations still experience major failures despite knowing these practices exist.\\n\\n\\
1n## Overall Reflection\\n\\nMy contributions became more focused as the discussion progressed. The initial post established the main argument, the peer responses broadened it across platform and manufacturing examples, and the summary post drew these together into a clearer position on Industry 5.0, resilience and operational accountability.\\n\\nI was most effective when connecting technical incidents to wider organisational and stakeholder consequences, framing outages, cyber incidents and data problems as socio-technical risks involving customers, workers, suppliers, governance and trust rather than purely technical failures.\\n\\nI was less effective at challenging arguments than extending them. My responses added safeguards and examples but did not always test feasibility, trade-offs or implementation limits.\\n\\nThe lesson I would take forward is that Industry 5.0 is a useful framing, but it should not become a checklist of human-centric and resilient design principles. The practical challenge is how those principles survive in real organisations where automation, efficiency and commercial pressure often push in the opposite direction."},{"id":"airbnb-nyc-demand-proxy-rationale","metadata":{"permalink":"/machine-learning/airbnb-nyc-demand-proxy-rationale","source":"@site/machine-learning/2026-05-11-demand-proxy-rationale.md","title":"Airbnb NYC Dataset - Demand Proxy Rationale","description":"Google Colab notebook Download .ipynb","date":"2026-05-11T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"MSc","permalink":"/machine-learning/tags/m-sc"},{"inline":true,"label":"Data Analysis","permalink":"/machine-learning/tags/data-analysis"},{"inline":true,"label":"Unit 1","permalink":"/machine-learning/tags/unit-1"}],"readingTime":3.06,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"},"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"airbnb-nyc-demand-proxy-rationale","title":"Airbnb NYC Dataset - Demand Proxy Rationale","authors":["rossbulat"],"tags":["Machine Learning","MSc","Data Analysis","Unit 1"]},"unlisted":false,"prevItem":{"title":"Collaborative Discussion 1 Reflection","permalink":"/machine-learning/collab-1-reflection"},"nextItem":{"title":"Data Analysis of Correlation and Regression Notebooks","permalink":"/machine-learning/data-analysis-of-correlation-and-regression-notebooks"}},"content":"**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/19S-n-be7Yk2uKEbtmSGr27mXciMSyHUb) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-05-11-demand-proxy-rationale.ipynb)\\n\\nThe [Airbnb NYC 2019 dataset](https://www.kaggle.com/code/whyalwaysme/ab-nyc-2019) is useful for exploring listing-level patterns, but it has one important limitation: **it does not contain direct demand measures**. There are no actual booking counts, occupancy rates, revenue figures, or guest ratings that can be used as a clean target variable.{/*truncate*/}\\n\\nThe notebook therefore tests whether `reviews_per_month` can act as a practical proxy for guest demand. It does not treat reviews as equivalent to bookings; the question is whether review activity behaves like a meaningful demand-related signal that can support later modelling work.\\n\\n## Why a proxy was needed\\n\\nA machine learning model needs a target variable. Here, the business question is whether Airbnb listing features can help identify listings that appear to attract stronger guest activity. However, the dataset only provides indirect indicators of activity, such as:\\n\\n- `number_of_reviews`\\n- `last_review`\\n- `reviews_per_month`\\n- `availability_365`\\n\\nOf these, `reviews_per_month` is the most useful starting point because it provides a normalised measure of ongoing review activity. Listings with no monthly review activity were assigned a value of zero, which keeps the proxy measurable across the full dataset.\\n\\n## Creating the review-activity proxy\\n\\nI created a new binary flag called `review_activity_proxy`.\\n\\nI calculated it by:\\n\\n1. Filling missing `reviews_per_month` values with zero.\\n2. Calculating the 75th percentile of the filled review-rate distribution.\\n3. Flagging listings at or above that threshold as proxy-positive.\\n\\nThe resulting proxy identifies listings with relatively high review activity compared with the rest of the dataset. It creates a reproducible success flag that can later be used as a classification target.\\n\\nThis is still only a proxy. A listing may receive more reviews because it has more bookings, but reviews are also affected by guest behaviour, host practices, listing age, and other factors. For that reason, the notebook treats the proxy as a preliminary demand-related signal rather than a direct measure of true demand.\\n\\n## Checking whether the proxy makes business sense\\n\\nI then checked whether the proxy behaved in a plausible way. A useful proxy should not look random or disconnected from the rest of the dataset.\\n\\nI checked this in two ways.\\n\\nThe spatial plot uses longitude and latitude to show whether proxy-positive listings appear within recognisable Airbnb market areas across New York City rather than being distributed without structure.\\n\\nI also explored listing-title length by cleaning the `name` field, calculating title length, grouping titles into bands, and comparing proxy-positive rates across those bands. This tests whether the proxy relates to listing presentation as well as location.\\n\\nNeither check validates demand on its own. Their purpose is to see whether the proxy has interpretable relationships with other listing fields rather than behaving like an arbitrary review count.\\n\\n## Why this matters for later modelling\\n\\nThe later modelling task depends on the credibility of the target variable. If that target were arbitrary, even a technically strong model would have limited business value.\\n\\nBy showing that the proxy is:\\n\\n- derived from an existing numeric field,\\n- consistently reproducible,\\n- linked to spatial listing patterns,\\n- related to title-length patterns, and\\n- interpretable within the Airbnb business context,\\n\\nthe notebook provides a stronger rationale for using `reviews_per_month` as a preliminary demand-related success flag.\\n\\n## Key takeaway\\n\\nWithout confirmed booking or occupancy data, the notebook cannot prove true Airbnb demand. However, `reviews_per_month` is measurable, reproducible and connected to observable listing characteristics, making it a defensible preliminary target for this exploratory project. Any later model should still describe it as a proxy rather than evidence of confirmed demand."},{"id":"data-analysis-of-correlation-and-regression-notebooks","metadata":{"permalink":"/machine-learning/data-analysis-of-correlation-and-regression-notebooks","source":"@site/machine-learning/2026-05-10-unit-3-activity.md","title":"Data Analysis of Correlation and Regression Notebooks","description":"Notebook 1: Covariance, Pearson Correlation and Regression","date":"2026-05-10T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"Regression","permalink":"/machine-learning/tags/regression"},{"inline":true,"label":"Data Analysis","permalink":"/machine-learning/tags/data-analysis"},{"inline":true,"label":"Unit 3","permalink":"/machine-learning/tags/unit-3"}],"readingTime":11.77,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"}
1,"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"data-analysis-of-correlation-and-regression-notebooks","title":"Data Analysis of Correlation and Regression Notebooks","authors":["rossbulat"],"tags":["Machine Learning","Regression","Data Analysis","Unit 3"]},"unlisted":false,"prevItem":{"title":"Airbnb NYC Dataset - Demand Proxy Rationale","permalink":"/machine-learning/airbnb-nyc-demand-proxy-rationale"},"nextItem":{"title":"Exploratory Data Analysis with Auto MPG Dataset","permalink":"/machine-learning/exploratory-data-analysis-auto-mpg"}},"content":"## Notebook 1: Covariance, Pearson Correlation and Regression\\n\\n**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1fbbKpTYXaxp-yXg_FaJmuM2YoeT_yudK) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-05-10-covariance-pearson-correlation-regression.ipynb)\\n\\n### Purpose of the notebook\\n\\nI expanded a basic correlation and regression example into a set of controlled experiments exploring how dataset properties affect statistical results.{/* truncate */} I varied sample size, noise, outliers, non-linearity, and data range, calculating and visualising correlations, regressions, and R\xb2 for each.\\n\\n### Changes and experiments\\n\\nI first made the notebook reusable by adding helper functions instead of writing all the calculations in one code block.\\n\\nFor the sample-size experiment, I used 10, 30, 100 and 1,000 points. Small datasets produced unstable estimates because individual points heavily influenced the slope and correlation, while larger samples stabilised the relationship.\\n\\nIncreasing the noise spread out the scatter plot, reduced the correlation, and lowered R\xb2, showing how noisy data obscures genuine relationships.\\n\\nWhen I added outliers, extreme values shifted the regression line noticeably, especially when they were far from the main cluster.\\n\\nThe non-linear experiment produced a clear pattern in the plot, but Pearson correlation and linear regression did not describe it well. This demonstrated that low correlation may reflect an unsuitable method rather than the absence of a relationship.\\n\\nFinally, restricting the input range made the relationship appear weaker. This demonstrated that models trained on narrow data do not generalise well to real-world variation.\\n\\n### What the results showed\\n\\nCorrelation and regression results depend heavily on dataset conditions. More data points made estimates more reliable; more noise obscured relationships; outliers distorted the fitted line; non-linearity limited linear methods; restricted ranges masked real patterns.\\n\\n| Experiment | n | Pearson r | R\xb2 | Notes |\\n|---|---:|---:|---:|---|\\n| Baseline (linear, \u03c3=10) | 1,000 | 0.89 | 0.79 | Clear positive trend |\\n| Small sample | 10 | 0.95 | 0.91 | Looks strong but unstable across reseeds |\\n| Large sample | 1,000 | 0.89 | 0.79 | Stable estimate |\\n| Low noise (\u03c3=2) | 1,000 | 0.99 | 0.99 | Almost deterministic |\\n| High noise (\u03c3=50) | 1,000 | 0.36 | 0.13 | Relationship largely obscured |\\n| Clean data | 100 | 0.89 | 0.79 | Reference for outlier test |\\n| With 3 high-leverage outliers | 103 | 0.21 | 0.04 | Outliers collapse the fit |\\n| Curved (y \u2248 x\xb2 + noise) | 300 | -0.05 | 0.00 | Linear methods miss the pattern |\\n| Full x range | 500 | 0.78 | 0.61 | Reference for range test |\\n| Restricted x range | 500 | 0.29 | 0.09 | Narrow range hides the trend |\\n\\nThe results demonstrated why statistics should not be interpreted in isolation. Scatter plots mattered just as much as the numbers because they revealed the structure of the data. In practice, good analysis combines statistical measures with visual inspection and careful documentation.\\n\\n### Analysis and conclusions\\n\\nThe results show that statistical outputs cannot be interpreted in isolation from the data that produced them. Sample size, noise, outliers, non-linearity, and the range of observed values each changed correlation strength and regression fit, even though the underlying generative relationship was held constant.\\n\\nThe implications also extend beyond the numbers. Models used in sensitive areas (finance, healthcare, recruitment) need reliable data to produce defensible predictions. Restricted datasets can create systems that work for some groups but not others. Outlier handling is a judgement call: unusual points may be errors, but they may also represent important rare cases. In any setting, assumptions and limitations should be documented.\\n\\n### Critical reflection\\n\\nWhat stood out from the notebook was that data analysis is not purely a technical task. A high correlation value or a neat regression line can be misleading if the dataset is too small, too narrow or affected by unexamined bias. Likewise, a low correlation value can be misleading if the relationship is non-linear. Machine learning professionals therefore need to understand the context of the dataset before choosing or evaluating an algorithm.\\n\\nThe notebook also shows how basic statistical methods can raise important professional questions, and why dataset quality, representativeness and transparency are central to responsible machine learning.\\n\\n---\\n\\n## Notebook 2: Linear Regression\\n\\n**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1uAWzBbGdGdFAuCXwvPS0rzvZ9w0Dr8KQ) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-05-10-linear-regression.ipynb)\\n\\n### Purpose of the notebook\\n\\nStarting with a 13-point baseline model, I explored how changes to the dataset affected linear regression. I varied the sample size, added consistent points, introduced noise and outliers, and restricted the range. For each variation, I recalculated the correlation, refitted the regression line, and observed how the predictions changed.\\n\\n### Changes and experiments\\n\\nI tested five variations of the original 13-point dataset:\\n\\n**Sample size:** Repeated random sampling from a simulated population showed the slope\'s standard deviation falling from ~1.13 (n=5) to ~0.16 (n=100). With only 5 points, the fitted line looked plausible but varied widely across resamples; with 100, it stabilised.\\n\\n**Trend-consistent points:** Adding 10 points that followed the existing negative pattern strengthened the correlation (-0.76 \u2192 -0.88) and raised R\xb2 (0.58 \u2192 0.78). Prediction at x=10 stayed essentially unchanged (~85.5).\\n\\n**Noisy data:** Adding inconsistent points effectively destroyed the relationship: Pearson r flipped to +0.04 and R\xb2 collapsed to 0.001. The algorithm still produced a line and prediction, but both were meaningless.\\n\\n**Outlier:** A single contradictory point (x=18, y=120) almost completely wiped out the negative trend (r: -0.76 \u2192 -0.10, R\xb2: 0.58 \u2192 0.009). Outliers deserve investigation; they may be errors or represent important rare cases.\\n\\n**Restricted range:** Using only x \u2264 8 produced *stronger* local statistics (r=-0.90, R\xb2=0.82) but the predicted value at x=10 drifted to 76.4 (from 85.6 baseline). High correlation did not guarantee reliable extrapolation.\\n\\n### What the results showed\\n\\nThe results changed sharply with the quality of the data. Consistent data strengthened relationships, noise obscured them, outliers distorted them, and narrow ranges masked extrapolation problems. The algorithm could not detect when the data was unsuitable; it generated predictions regardless.\\n\\n| Experiment | n | Pearson r | R\xb2 | Predict at x=10 |\\n|---|---:|---:|---:|---:|\\n| Original | 13 | -0.76 | 0.58 | 85.6 |\\n| + Trend-consistent points | 23 | -0.88 | 0.78 | 85.5 |\\n| + Noisy points | 23 | +0.04 | 0.001 | 89.6 |\\n| + 1 Outlier (18, 120) | 14 | -0.10 | 0.009 | 91.5 |\\n| Restricted range (x \u2264 8) | 8 | -0.90 | 0.82 | 76.4 |\\n\\n### Analysis and conclusions\\n\\nThe clearest finding is that data quality matters as much as algorithm choice. The same `LinearRegression` call produced a reasonable model on the original data, a stronger model when trend-consistent points were added, and an essentially meaningless model once noise or a single contradictory outlier was introduced. Restricting the range produced the most subtle failure mode: the local fit *improved* (r=\u22120.90) while the prediction at x=10 drifted by nearly 10 units, illustrating that a high in-sample R\xb2 does not guarantee reliable extrapolation.\\n\\nIn practical settings such as loan scoring, pricing and risk assessment, regression models trained on biased, incomplete or unrepresentative data will still produce confident predictions. Those predictions will still be acted on, even though the algorithm cannot tell the operator when its inputs are unsuitable.\\n\\n### Critical reflection\\n\\nRegression is fundamentally about fitting a line to a specific dataset. A high R\xb2 might seem to validate the model, but statistics cannot guarantee practical utility. Professional responsibility lies in understanding data limitations, validating assumptions, investigating anomalies, and acknowledging the conditions that predictions depend on.\\n\\n---\\n\\n## Notebook 3: Multiple Linear Regression\\n\\n**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1wVxw8Tqx0yXVtOH0pTsikxXIIKeDn3tC) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-05-10-multiple-linear-regression.ipynb)\\n\\n### Purpose of the notebook\\n\\nUsing a small car dataset, I explored how changes to the data affected multiple linear regression. I predicted CO2 emissions from Weight and Volume, then tested different sample sizes, trend-consistent points, noisy data, outliers, and restricted ranges to see how each change affected correlations, coefficients, R\xb2, and predictions.\\n\\n### Changes and experiments\\n\\nI added helper functions and expanded the original single-prediction example into a series of controlled experiments:\\n\\n**Baseline:** Fitted a model with Weight and Volume predicting CO2, establishing baseline correlations (Weight\u2013CO2: 0.55, Volume\u2013CO2: 0.59, R\xb2: 0.38). Predicted CO2 at Weight=2300, Volume=1300 \u2248 107.2 g.\\n\\n**Fewer data points:** Reduced to 12 samples; estimates became unstable, correlations weakened toward zero (Weight\u2013CO2 even flipped sign), and R\xb2 dropped to ~0.18.\\n\\n**Trend-consistent points:** Added points following the expected pattern, strengthening correlations (~0.62 / 0.64) and lifting R\xb2 to ~0.44.\\n\\n**Noisy data:** Added conflicting points that reversed relationships, collapsing R\xb2 to ~0.02, showing how poor data quality can mask real patterns.\\n\\n**Outlier:** One unusual point pulled coefficients sharply, dropping R\xb2 to ~0.04 and weakening both correlations.\\n\\n**Restricted range:** Using only lower weight values produced moderate local statistics (R\xb2 ~0.22) but unreliable extrapolation to heavier cars.\\n\\n### What the results showed\\n\\nThe model\'s apparent reliability shifted across the experiments as data quality, sample size, and range coverage changed. R\xb2 ranged from ~0.02 to ~0.44, yet the algorithm still produced confident predictions in every case and could not signal when the data was unsuitable.\\n\\n| Experiment | n | Corr W\u2013CO2 | Corr V\u2013CO2 | R\xb2 | Pred CO2 (2300/1300) |\\n|---|---:|---:|---:|---:|---:|\\n| Baseline | 36 | 0.55 | 0.59 | 0.38 | 107.2 |\\n| Fewer points | 12 | -0.17 | 0.08 | 0.18 | 68.6 |\\n| + Trend-consistent | 41 | 0.62 | 0.64 | 0.44 | 109.3 |\\n| + Noisy points | 41 | 0.06 | 0.13 | 0.02 | 95.6 |\\n| + Outlier | 37 | 0.17 | 0.20 | 0.04 | 105.5 |\\n| Restricted (low weight) | 18 | 0.25 | 0.43 | 0.22 | 85.1 |\\n\\n### Analysis and conclusions\\n\\nMultiple linear regression appears deceptively definitive: two coefficients, an intercept, an R\xb2. Yet every one of those numbers shifted substantially across the experiments. Even the baseline R\xb2 of 0.38 already signals that Weight and Volume alone are incomplete predictors of CO2; fuel type, engine efficiency, vehicle age and driving conditions are all absent from the dataset. This omission of relevant predictors is what limits the model\'s usefulness.\\n\\nModels like this are routinely used in emissions reporting, insurance pricing and regulatory compliance. A practitioner deploying one would need to justify the choice of predictors, document how the dataset was assembled, and be transparent about the proportion of variation the model leaves unexplained.\\n\\n### Critical reflection\\n\\nThe multiple-regression notebook showed that regression is fundamentally estimation; it is curve-fitting adapted to a specific dataset. The models were too small and simplified for real decision-making. Their value lies in showing why professional judgement matters as much as technical execution: understanding data limitations, investigating anomalies, documenting assumptions, and refusing to overstate what statistics can prove.\\n\\n---\\n\\n## Notebook 4: Polynomial Regression\\n\\n**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1H_KzNvdycLnF4vpIcuCXNL8k5OY6mrXc) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-05-10-polynomial-regression.ipynb)\\n\\n### Purpose of the notebook\\n\\nUsing tollbooth data with a curved relationship between time of day and vehicle speed, I explored how changes to the dataset affected polynomial regression. I fitted polynomial models, then varied the sample size, added trend-consistent and noisy data, introduced outliers, restricted the range, and compared polynomial degrees to see how each change affected reliability.\\n\\n### Changes and experiments\\n\\nI added helper functions and expanded the original third-degree polynomial example into controlled experiments:\\n\\n**Baseline:** Fitted a degree-3 polynomial to predict speed at 17:00 (\u224888.9). R\xb2 was 0.94 but Pearson r was only 0.43, because the relationship is curved, not linear.\\n\\n**Fewer data points:** Used every other observation. R\xb2 stayed high (~0.96) and the prediction barely moved (~88.5), but the model was based on much less evidence and would be more vulnerable to sampling bias.\\n\\n**Trend-consistent points:** Added points following the curved pattern; R\xb2 stayed at ~0.95 and predictions remained stable.\\n\\n**Noisy data:** Added conflicting points that distorted the curve, dropping R\xb2 to ~0.65 and pulling the 17:00 prediction down to ~81.\\n\\n**Outlier:** One extreme value (x=11, y=130) pulled the curve sharply, collapsing R\xb2 to ~0.39 and shifting predictions to ~92.\\n\\n**Restricted range:** Using only 05:00\u201316:00 data produced strong local fit (R\xb2 ~0.94) but the 17:00 prediction dropped to ~77, showing how extrapolation outside the observed range becomes unreliable.\\n\\n**Polynomial degrees:** Compared degrees 1 through 6. Degree 1 underfitted (R\xb2=0.18); degree 3 captured the pattern well (R\xb2=0.94); degree 6 fitted training data even more closely (R\xb2=0.98) but risked overfitting noise rather than learning generalisable structure.\\n\\n### What the results showed\\n\\nPolynomial regression handled curved patterns effectively, but the results were highly sensitive to data quality and model complexity. Predictions changed across the experiments, while Pearson correlation alone proved insufficient for evaluating curved relationships. Outliers and noise distorted the curves, restricted ranges masked extrapolation problems, and higher-degree polynomials improved the training fit at the risk of overfitting.\\n\\n| Experiment | n | Pearson r | R\xb2 | RMSE | Predict at x=17 |\\n|---|---:|---:|---:|---:|---:|\\n| Baseline (degree 3) | 18 | 0.43 | 0.94 | 3.5 | 88.9 |\\n| Fewer points (every other) | 9 | 0.26 | 0.96 | 2.9 | 88.5 |\\n| + Trend-consistent | 22 | 0.49 | 0.95 | 3.2 | 88.9 |\\n| + Noisy points | 22 | 0.22 | 0.65 | 9.2 | 81.1 |\\n| + 1 Outlier (11, 130) | 19 | 0.33 | 0.39 | 14.5 | 92.3 |\\n| Restricted (05:00\u201316:00) | 11 | 0.95 | 0.94 | 2.0 | 77.1 |\\n\\n**Polynomial degree comparison (original data):**\\n\\n| Degree | R\xb2 | RMSE | Predict at x=17 |\\n|---:|---:|---:|---:|\\n| 1 | 0.18 | 13.4 | 83.9 |\\n| 2 | 0.76 | 7.3 | 81.2 |\\n| 3 | 0.94 | 3.5 | 88
1.9 |\\n| 4 | 0.95 | 3.2 | 87.2 |\\n| 6 | 0.98 | 2.1 | 84.3 |\\n\\n### Analysis and conclusions\\n\\nThe degree comparison is the clearest illustration of the bias and variance trade-off in this set of notebooks. A degree-1 model underfits the curved tollbooth pattern (R\xb2=0.18) and a degree-6 model achieves the best training R\xb2 (0.98), but the degree-6 fit is shaped as much by noise as by the underlying pattern. This is visible in how its 17:00 prediction drifts to 84.3 while the more conservative degree-3 model predicts 88.9. The right choice is the simplest model that captures the structure, not the one with the highest R\xb2.\\n\\nThe other experiments echo points seen in the linear cases (outliers and noisy points distort the curve, restricted ranges break extrapolation), but they land harder here because polynomial models are more flexible and therefore more sensitive to each of these problems. In a real traffic-prediction setting, an overfitted model deployed against limited or biased data could produce confidently wrong outputs that feed into planning or safety decisions.\\n\\n### Critical reflection\\n\\nThe polynomial experiment showed that more flexible models are not automatically better. Polynomial regression handles curves that linear regression cannot, but it is also more sensitive to outliers and prone to overfitting. Model selection requires balancing expressiveness with reliability while accounting for the limits of the data. High R\xb2 does not guarantee practical utility, especially when data is limited, noisy, or used beyond its observed range."},{"id":"exploratory-data-analysis-auto-mpg","metadata":{"permalink":"/machine-learning/exploratory-data-analysis-auto-mpg","source":"@site/machine-learning/2026-05-09-auto-mpg.md","title":"Exploratory Data Analysis with Auto MPG Dataset","description":"Google Colab notebook Download .ipynb","date":"2026-05-09T00:00:00.000Z","tags":[{"inline":true,"label":"Machine Learning","permalink":"/machine-learning/tags/machine-learning"},{"inline":true,"label":"Introduction","permalink":"/machine-learning/tags/introduction"},{"inline":true,"label":"Unit 2","permalink":"/machine-learning/tags/unit-2"}],"readingTime":8.09,"hasTruncateMarker":true,"authors":[{"name":"Ross Bulat","title":"Full Stack Engineer","url":"https://github.com/rossbulat","page":{"permalink":"/machine-learning/authors/rossbulat"},"socials":{"x":"https://x.com/rossbulat","github":"https://github.com/rossbulat","linkedin":"https://www.linkedin.com/in/rossbulat/"},"imageURL":"https://jkrb.io/img/rossBulat.webp","key":"rossbulat"}],"frontMatter":{"slug":"exploratory-data-analysis-auto-mpg","title":"Exploratory Data Analysis with Auto MPG Dataset","authors":["rossbulat"],"tags":["Machine Learning","Introduction","Unit 2"]},"unlisted":false,"prevItem":{"title":"Data Analysis of Correlation and Regression Notebooks","permalink":"/machine-learning/data-analysis-of-correlation-and-regression-notebooks"}},"content":"**Google Colab notebook:** [Open in Google Colab](https://colab.research.google.com/drive/1P-_U8k6PZkqrnI1KQ_KgakW7T-2v_rGZ) \xb7 **Raw Notebook:** [Download `.ipynb`](/notebooks/2026-05-09-auto-mpg.ipynb)\\n\\n## Introduction\\n\\nI used exploratory data analysis (EDA) to assess whether the Auto MPG dataset was suitable for machine learning and to identify key uncertainties before modelling.{/* truncate */}\\n\\nI followed a standard EDA workflow: importing the analysis libraries, loading the data, inspecting its structure and summary statistics, separating numeric and categorical variables, checking for missing values, and using visualisations (including heat maps and scatter plots) to examine relationships.\\n\\nThe dataset contains **398 rows** and **9 original columns**. The main variables are fuel economy (`mpg`), engine size (`displacement`), engine power (`horsepower`), vehicle weight, acceleration, model year, origin, and car name.\\n\\n## 1. Loading and inspecting the data\\n\\nI began by loading `auto-mpg.csv` into pandas and reviewing the first and last rows, dataset shape, and inferred data types. This initial check exposed an important quality issue: `horsepower` had been read as an object c
1olumn instead of a numeric variable.\\n\\nThis was because missing values were represented by a question mark (`?`) rather than standard `NaN`. It was a useful reminder that missing data is not always immediately visible through `isna()` on raw input, especially when pandas has inferred a text type.\\n\\n## 2. Identifying missing values\\n\\nAfter converting `horsepower` using `pd.to_numeric(..., errors=\'coerce\')`, the question marks were correctly converted to missing values. The missing-value summary was:\\n\\n| | missing_count | missing_percent |\\n|:-------------------|----------------:|------------------:|\\n| horsepower | 6 | 1.51 |\\n| mpg | 0 | 0 |\\n| cylinders | 0 | 0 |\\n| displacement | 0 | 0 |\\n| weight | 0 | 0 |\\n| acceleration | 0 | 0 |\\n| model year | 0 | 0 |\\n| origin | 0 | 0 |\\n| car name | 0 | 0 |\\n| horsepower_imputed | 0 | 0 |\\n| origin_label | 0 | 0 |\\n| origin_America | 0 | 0 |\\n| origin_Europe | 0 | 0 |\\n| origin_Japan | 0 | 0 |\\n\\n\\n\\nThe chart confirms that the missing values are confined to `horsepower`. This left a small but non-trivial missing-data issue: **six missing `horsepower` values** (approximately **1.51%** of records). To keep the workflow transparent, I retained the original `horsepower` column and created a separate `horsepower_imputed` feature using median imputation for analysis that required complete numeric inputs.\\n\\nI selected median imputation because it is straightforward, reproducible, and generally more robust to skew than mean imputation.\\n\\n## 3. Encoding categorical values\\n\\nThe task required categorical values to be represented numerically. In this dataset, `origin` was already stored as integer codes (America = 1, Europe = 2, Japan = 3). However, integer encoding (1, 2, 3) implies an artificial order and distance between what are actually nominal categories\u2014there is no meaningful sense in which Japan is \\"three times\\" America, or in which Europe sits \\"between\\" the other two.\\n\\nFor this reason, I first mapped the integer codes to readable labels and then applied **one-hot encoding** using `pd.get_dummies(...)`, which produced three binary indicator columns:\\n\\n| Original origin | origin_America | origin_Europe | origin_Japan |\\n|---|---:|---:|---:|\\n| America | 1 | 0 | 0 |\\n| Europe | 0 | 1 | 0 |\\n| Japan | 0 | 0 | 1 |\\n\\nOne-hot encoding avoids the false ordinal assumption that integer encoding can introduce, and it is generally the more appropriate choice for nominal categorical features in machine learning pipelines. Each indicator column is itself a valid numeric feature (a Bernoulli 0/1 variable), so unlike a single arbitrary integer code, the one-hot columns can legitimately appear in a Pearson correlation matrix against `mpg` and other continuous variables.\\n\\nThe origin counts were:\\n\\n| origin_label | count |\\n|:---------------|--------:|\\n| America | 249 |\\n| Japan | 79 |\\n| Europe | 70 |\\n\\n## 4. Skewness and kurtosis\\n\\nI calculated skewness and kurtosis for the continuous numeric variables only. Skewness indicates asymmetry in a distribution, while kurtosis indicates tail behaviour relative to a normal distribution. The one-hot `origin_*` columns are excluded from this analysis because they are binary indicator variables. Distribution shape statistics like skewness and kurtosis are not informative for 0/1 features (their shape is fully described by their proportion).\\n\\n| | skewness | kurtosis |\\n|:-------------------|-----------:|-----------:|\\n| mpg | 0.457 | -0.511 |\\n| cylinders | 0.527 | -1.377 |\\n| displacement | 0.72 | -0.747 |\\n| horsepower | 1.087 | 0.697 |\\n| horsepower_imputed | 1.106 | 0.764 |\\n| weight | 0.531 | -0.786 |\\n| acceleration | 0.279 | 0.419 |\\n| model year | 0.012 | -1.181 |\\n\\n\\n\\nThe histogram makes the right skew in `horsepower_imputed` clear. Most vehicles sit in the low-to-mid power range, with a smaller number of high-powered vehicles forming the right tail. `displacement`, `weight`, and `cylinders` also showed positive skew, which aligns with a dataset dominated by moderate vehicles and fewer extreme large-engine observations.\\n\\nMost kurtosis values were close to zero or negative, so the distributions were not generally dominated by heavy tails. Since skew and outliers can affect algorithm performance and interpretation, later stages may require transformation, scaling, or more robust estimators depending on the model chosen.\\n\\n## 5. Correlation heat map\\n\\
1nI used a correlation heat map to examine linear relationships between numeric features. Because `origin` is represented as three binary one-hot columns, each can be included in the correlation matrix as a point-biserial-style relationship with `mpg`. This is a valid use of Pearson correlation between a continuous variable and a 0/1 indicator.\\n\\nThe correlations with `mpg` for the continuous variables were:\\n\\n| | mpg |\\n|:-------------------|-------:|\\n| mpg | 1 |\\n| model year | 0.579 |\\n| acceleration | 0.42 |\\n| horsepower_imputed | -0.773 |\\n| cylinders | -0.775 |\\n| displacement | -0.804 |\\n| weight | -0.832 |\\n\\n\\n\\nThe strongest negative relationship was between `mpg` and `weight`, which is intuitive: heavier vehicles tend to consume more fuel. `displacement`, `horsepower`, and `cylinders` were also strongly negatively associated with `mpg`.\\n\\nThe heat map also shows strong correlations between several mechanical variables. `displacement` and `weight` have a correlation of **0.93**, while `cylinders` and `displacement` have a correlation of **0.95**. This degree of multicollinearity would need to be considered when selecting and interpreting a model, even if the main aim were simply to predict `mpg`.\\n\\n`model year` showed a positive relationship with `mpg`, suggesting that newer vehicles in this historical sample were generally more fuel efficient. I treated this carefully in interpretation, since correlation alone does not establish causation and may reflect broader historical factors such as regulation, design shifts, or market priorities.\\n\\nThe one-hot `origin_*` columns showed the direction expected from the grouped summaries in Section 7: `origin_America` was negatively correlated with `mpg`, while `origin_Japan` and `origin_Europe` were positively correlated with `mpg`. Because the three indicators are mutually exclusive and sum to 1 for every row, they are not independent of each other, so they should be interpreted together rather than as fully separate features.\\n\\n## 6. Scatter plots\\n\\nThe scatter plots focused on pairwise relationships involving `mpg`, particularly:\\n\\n- `mpg` vs `weight`\\n- `mpg` vs `horsepower_imputed`\\n- `mpg` vs `displacement`\\n- `mpg` vs `acceleration`\\n- `mpg` vs `model year`\\n\\n\\n\\nThe `weight` plot shows this pattern most clearly: `mpg` falls as vehicles become heavier, and the fitted line highlights the inverse relationship. The same downward trend appeared as `horsepower_imputed` and `displacement` increased. Colouring the points by origin also revealed clusters in different parts of the feature space. In this dataset, American-origin vehicles appeared more frequently in heavier, larger-displacement regions, while Japanese and European vehicles were more common in lighter, higher-mpg regions.\\n\\nThese patterns are useful for modelling, but need to be interpreted with caution. Origin may be acting as a proxy for other factors (vehicle size, production era, market segment, or design choices), so it should not be treated as a standalone causal explanation.\\n\\n## 7. Grouped summaries\\n\\
1nI also compared averages by origin:\\n\\n| origin_label | count | avg_mpg | avg_weight | avg_horsepower |\\n|:---------------|--------:|----------:|-------------:|-----------------:|\\n| Japan | 79 | 30.45 | 2221.23 | 79.84 |\\n| Europe | 70 | 27.89 | 2423.3 | 80.93 |\\n| America | 249 | 20.08 | 3361.93 | 118.64 |\\n\\n\\n\\nThe box plots provide more context. American vehicles had the lowest median MPG and Japanese vehicles the highest, although the distributions overlapped and there was variation within every group. In this sample, vehicles from Japan and Europe also tended to have lower average weight and horsepower. This was a useful example of how EDA methods work together: correlation gives a quantitative overview, while grouped summaries and distributions improve interpretability.\\n\\n## 8. Reflection against the learning outcomes\\n\\nAt face value, Auto MPG appears straightforward, but the `horsepower` issue demonstrated why EDA is essential before any model development. Hidden missingness in a nominally numeric feature can directly undermine model reliability.\\n\\nThe EDA highlighted several issues to address before selecting a model, including missing values, skewed feature distributions, multicollinearity among mechanical variables, and encoding decisions for categorical data. Each has implications for model validity, interpretability, and defensibility.\\n\\nAlthough this dataset is less sensitive than personal or clinical data, there are still professional and ethical considerations. It is historical data, so findings should not be overgeneralised to contemporary vehicle populations. Similarly, origin-based comparisons should be framed carefully to avoid simplistic conclusions, since origin may co-vary with broader engineering and market factors.\\n\\nI also treated reproducibility as part of professional practice by documenting cleaning decisions, preserving raw and transformed columns, and structuring the analysis so another team member could review the assumptions and continue the workflow into modelling.\\n\\n## Conclusion\\n\\nOverall, this EDA suggests that the Auto MPG dataset is suitable for introductory machine learning tasks, particularly MPG prediction, but only after data preprocessing. The hidden missing values in `horsepower`, skew in engine-related variables, strong inter-feature correlations, and categorical encoding choices all require explicit handling.\\n\\nThe exercise reinforced the value of EDA as a way to reduce risk before modelling. It helps identify data quality problems early, supports transparent methodological decisions, and provides a stronger foundation for responsible model development."}]}}')}}]);
Line numbers count LF bytes from the start of the resource, as the search results do. Vendor segments are library code the classifier recognised; they are stored but not indexed. Bytes are shown as Latin1 characters, one per byte.