AI: Here and Now
I've written a few times about AI and my various thoughts on/around it, however as the underlying technology has advanced at a stellar pace (and as it becomes a part of everyday life for many people both in and out of work), it seemed time to write about my own journey (from where I began, to where I am now).
If I wind the clock back far enough, I recall when large companies began rebranding machine learning as AI to better align with market trends (or in plain speak, to sell more of the same product). Back then, the technology hadn't evolved to a great extent and depending on the context its practical use was great for very specific scenarios with clean datasets, but middle of the road at best for your average (especially those with inaccuracies). My biggest frustration with ML was the lack of explainable decision logic, as attempting to explain to clients why a model had flagged a specific entity as a threat despite no clear evidence was always a headache nobody wanted.
Wind the clock forward a little and we get to the rise of large language models (LLMs for short), which (in my opinion) have become the largest technological shift in the last 10 years. Their initial incarnations were impressive at the time, but still limited in their capability (frequent loops and vivid hallucinations). Attempting to use these for anything other than simple tasks was often met with frustration (at least for me), and the information provided left a lot to be desired. My favourite example of this was seeing code being created where the basics of truthiness in Python wasn't properly understood (how times have changed).
It was Wendell from Level1Techs (a great YouTube channel btw) who mentioned how from a local AI perspective an inflection point had been reached. The example being how a local AI was now able to self-configure its own cluster, benchmark it, tune it, and document all of the steps taken. While this was very impressive (and local AI continues to grow in capability, even if purchasing a GPU currently requires selling a kidney), the capabilities of commercial AI had already passed that point and were far into the distance. The 'AI as a Service' had become a reality, not just for computing tasks, but as a personal companion.
At this point I want to be clear that even at time of writing, AI isn't perfect, hallucinations do still happen, and some of the advice provided can result in death. By no stretch of the imagination do I believe it to be the source of all truth, but that is the direction of travel, and it is getting better. That said, always verify what it tells you.
Back to the present day, and there are commercial AI models that can run most aspects of a business, and can run most aspects of your personal life. While I'm not sold on the latter (yet), AI for work is now a powerful tool that when used correctly (and responsibly) can accelerate your productivity while improving your overall accuracy. I'm fortunate in my current role to have access to a range of models, and while I wouldn't say any are perfect, knowing which ones to use for a specific task (and how to guide them to get the best results), can make the difference between a weeks worth of manual tasks or 1 hour (or less) spent achieving the same results (if not better).
Outside of work I'm using AI (in this context Claude Opus) to tackle some of the projects I've had on my backlog for a long time (in some cases literal years). While I'm always happy to lose an evening to some personal coding / growth, time is a finite resource and as with everything in life, some things always end up at the bottom of the todo list.
For this article I'm covering three projects (technically two coding projects and one very long/in-depth conversational thread to fix an ongoing technical headache), to provide a view on just how far they have evolved, and how the days of it being a complex autocomplete have passed.
The projects:
- A vehicle tracker I built custom firmware for many years ago
- A rework of a screensaver from my childhood
- An enterprise mobile repeater with headaches for days
The Vehicle Tracker
Many years ago I purchased a Freematics One+ device to act as a tracker for vehicle to show live telemetry/telematics with a software application I worked with. It's a tiny device (in comparison the something like the AutoPi) but wasn't without its share of issues:- The firmware was unstable (as in, really unstable)
- Some of the functionality never really worked correctly
- The wiring for the device didn't align with the documented specifications (the serial console was always a challenge)
- Expanding the firmware was complex due to its structure / compatibility with other devices
- Trying to switch between Arduino builder / PlatformIO / Visual Studio Code wasn't ideal
- It overheated (a lot), and required me to drill holes in the plastic case to keep it cool
While I made progress with the custom firmware many years ago and did a few successful demos to clients showing how for fleet management it could be a viable solution, the headaches outweighed the benefits and so it was stored away for a later date. About 12 months ago I tried to revive the project but attempting to get the latest firmware to work resulted in an unusable device that was worse than my original firmware. Attempting to get help for the device didn't work out either, and so back to the cupboard it went.
Fast forward to three weeks ago whereby I decided to see just how capable the latest Opus model is (this isn't an advertisement, just a record of how things progressed). I figured this would be a challenge considering the many firmware changes over the years (and the incompatibility my last attempt encountered), but why not...
The initial scope was simple: get a build of the latest firmware working on my device, whereby the OBD interface, the GPS, and the modem must function. No SD card, no WiFI or Bluetooth, no MEMS, just the absolute basics (as a starting point). With the device connected to the laptop and one long prompt later, Opus began to do its thing. It reviewed the public GitHub repository to get details of the firmware, and reviewed the product pages to get details of the different models. Within an hour it had built the firmware and flashed it to my device, and then, failure. This however, is where AI comes into its own...
With the device connected it checked the initial boot log which covered the low-level hardware, which is where the first oddity was discovered (as the specifications of my device didn't quite match the Model A specifications on their website) A little odd given the purchase order showed that's what I purchased, but either way it continued troubleshooting and created another build. Another upload triggered and the device booted a little further this time. A few more attempts and the device appeared to be getting somewhere, however the core components still weren't responding. This is where Opus questioned me regarding what model I actually had. It had already indicated that there were wiring differences between the Model A and Model B, but asked me to verify the exact board model / components if possible given the wiring seemed to be mismatched.
After opening the device I spotted the first inconsistency in that the modem was not the one listed on the website (mine is much older). I double-checked the order again and it was definitely the right model (I'm tired but not that tired). When I fed this back to Opus it reviewed the GitHub content again and discovered that the early versions of the Model A (which referenced my modem) had different wiring to the advertised version and interfaced with the components differently (it also explained why the bootloader reported slightly different hardware than expected). With this information at its disposal another firmware image was built and uploaded, and for the first time in years the device burst into life. GPS immediately worked, OBD was scanning for a vehicle, and the modem was reporting it needed a SIM card.
Why is the above important you might ask? I raised a similar question on GitHub during my prior attempt at getting the device working again, as I had a similar outcome with the latest firmware. The responses were few, but all pointed towards the official documentation / wiring diagrams for the Model A / Model B, and how I must be doing something wrong. Sadly for me, those diagrams didn't reflect my hardware due to the updated revision. It was AI that suggested there could be a mismatch based on the bootloader info, and it was AI that reviewed all of the commit history / the code and determined the earlier model config that had seemingly been obsoleted.
The issue that had stumped me previously was resolved in under an hour. Better still, it wrote a simple markdown document that contained the details of my device (including pin assignments) in case I (or it) needed to reference it again. My earlier device now became the unofficial Model 0, and a few more grey hairs can wait for another day.
With the initial build working (including testing on a vehicle to be sure), it was time to make it more challenging. Time for some more complex requirements:
- Review the current code (for the model-0) and check for any bugs / quirks / performance issues that should be fixed
- Restructure the code to split it into clean modules based on the physical hardware components (i.e., GPS / modem / WiFi / OBD etc)
As those using AI might say: "tokens well spent!". Taking aside the code refactor (which for the first time made it easy to follow, including the interactions between the components), the list of bugs was substantial. Some of the highlights:
- Core code not handling known hardware errata with the ESP32
- Multiple crash scenarios ranging from handling of the modem, OBD connection stalls/timeouts, and general use-after-free
- Performance challenges due to unoptimised code (remember that overheating I mentioned?)
- GPS inaccuracies due to floating point precision handling
- Power saving mode not always working (parasitic drain is not fun)
While all of the above is great, it was time to really test what Opus was capable of and to get the code to where I'd like it to be. Time for more complex tasks:
- Migrate the code to ESP-IDF 6.1 (the latest available version at time of writing)
- Remove all Arduino references / dependencies
- Remove all PlatformIO references / dependencies
- Remove any obscure code that isn't in use
- Perform a full refactor again (post-migration), optimise where possible, and keep the code tidy / easy to follow
- Add the base footprint for secure telemetry upload in real-time via the mobile connection
While the above got it to a really great place, there is still more that can be achieved (without burning tokens for the sake of it):
- Add support for the newer Model A / Model B variants, ensuring the same functionality across all models
- Automatically build firmware for the three models each time changes have been made / tests have been performed
- Add WiFi hotspot support (with the latest supported encryption standard)
- Create a lightweight web interface that leverages the ESP-IDF webserver component
- Get the web interface logging the various car details / PID values / debug logging (in real-time)
- Add encrypted SD card storage for both metrics and logging
- Add support for OTA firmware updates (split partition approach)
- Add support for ELM PID injection (for live testing)
As expected this took some time (hours rather than days), but it handled it better than expected and a few hours later I had firmware built for the various models, including a WiFi hotspot and simple / clean web interface that provides the vehicle stats in real-time (with a lightweight responsive design suitable for mobile and laptop).
One final challenge to get this project to the next level... Make it testable locally, with full simulation of all aspects of the vehicle (including a simulated journey, GPS positional changes, mobile signal variation etc). In truth this is the one where I expected it to trip up, but it went through it and actually got it done. Using the ESP-IDF version of the QEMU application, it created the various custom device handlers to replicate the GPIO pins/components used by the device, and then began working in a loop to get the firmware able to run both on a physical device and via QEMU.
When it finished, the firmware could be executed via a single command and allowed for local testing/development without the real device. More bugs were found during the process (both with QEMU itself, and edge-cases in the firmware that would have caught me out on the road), but it fixed those along the way. Kudos to Opus for just how much it achieved in such a short span of time (something that would have taken me weeks (if not months).
Johnny Castaway
The second project is a flashback to my childhood, focusing on a screensaver that used to run on a spare monitor to add something light-hearted into a room usually filled with broken parts and magic smoke (from my poor attempts at soldering). Released in 1992, it provided a story of a man stranded (mostly) on a desert island, and how he kept himself occupied.Over the years there have been a few attempts at recreating the screensaver so it can be ran either as a native application or a web page hosted by Node. All have their pros and cons, and unfortunately none of them aligned with my requirements. As I wanted to run it via a web browser on simple (low power) systems, hosted by a server that only serves static pages, and with the code optimised to prevent constant decoding of resources, it needed some work. I also wanted to see what it would look like if the graphics were modernised (while still keeping the original look/feel).
In half a day it has examined the original screensaver and built a clean version that can be hosted as a static website. Multiple rendering bugs had been identified and resolved (it used headless Firefox to analyse the rendering frame by frame). Finally, it began running other models (suited to image manipulation) to create a new set of reference images that keep the cartoon style but make it look more modern. The result is below (switched on/off via a hotkey) and adds a nice touch to it. Seeing the CPU usage reduction by optimising the rendering was also impressive and goes to show how AI really can work well with iterative development (when configured correctly, and when using a capable model).
Mobile Signal
The third project highlights a different use for AI: using it as a technical companion / specialist. Going down this road introduces its own concerns around safety / accuracy of advice, but as its a technical challenge being discussed its easier to double-check the conversational points and ensure the right advice is being given.In a nutshell, my house has near-zero mobile signal both indoors and outdoors (with indoors being worse due to the bricks / privacy film on the windows) and while WiFi calling is great (when it works) it doesn't help devices that don't support it (or the delivery driver at the front door). As signal boosters are a thing (and legal from an Ofcom standpoint providing you use an approved model), it seemed like a no-brainer.
One (ridiculously expensive) device later and everything should be good to go, except it wasn't. Taking aside the distributor sending me the wrong length cable (and with the wrong connector), initial results (even after tuning the antenna position) were awful. Three of the five supported bands were oscillating and even with that fixed, the upload speed would drop from around 10Mb/s at startup to around 0.1Mb/s after 30 minutes.
Using Claude as a sounding board I began describing the devices being used (providing links to the various pieces), and the behaviour I was seeing. After it had reviewed the various web pages / technical details it began asking targeted questions regarding its placement / the current signal strength (on the unit and on each mobile device) / what bands are actively being used by each connected device etc.
In what became a very long conversation (and at time of writing, one that is still ongoing), solid advice was provided regarding current signal strengths, which parts were likely struggling, and what actions should be taken to further diagnose / resolve the issue. Included in this was the pros and cons of different suggestions I'd made regarding upgrading parts / deploying additional shielding etc, all with solid feedback as to if they would likely work or if they would be an unwarranted expense. All the while, my support ticket with the manufacturer remained only answered with a holding email (with no progress after a week).
While the above is still ongoing, following the recommendations it provided the mobile coverage indoors has been improved and is now usable. Granted, it still isn't perfect and needs further tweaks to get it to its final state, but its a significant improvement from where it began. It also highlights the reasoning capability of the AI, and its ability to process information it hasn't seen before (not been trained on), and adjust its responses based on that.
Closing
So what's my conclusion with all of the above? I'd say we've reached a point whereby the use of AI for common / repeatable tasks has become commonplace and its usage will continue to grow (in line with how the models continue to improve / how affordable their prices are). There's no denying the value that it brings, and hopefully continues to bring moving forward.An analogy I find fitting is that of how lumber yards (and the tools that underpin their operation) changed the construction industry all those years ago. The introduction of tools including the band-saw / belt sanders / revolutionised how timber would be processed, and how it could be produced at a higher volume (more efficiently as well). There was a major shift in how operators in factories functioned (moving away from traditional hand tools to operating large machinery), and while staff did need to be retrained to use the newer equipment (while keeping their knowledge / experience on how to use the original tools), it was an industrial shift for the better (and that society now depends upon).
For me, understanding the underlying tools you use (how to use them safely, things to be aware of, design principals, and best practices) is still a core part that shouldn't be lost / overlooked. Being able to use newer tooling that revolutionises the process is a part of change (just like the previous industrial revolution), but it doesn't replace understanding the basics. While I do believe that relying on AI to create your dream application, without personally requiring any prior knowledge of coding / security / operational principals, is dangerous (a long list of compromised applications / services exist due to this), AI is getting better at handling these aspects as a default, and that over time the models / capabilities will evolve to where this is less of a concern / is automatically handled for you.
Let's see where this revolution takes us (and what my next project will be).