A friend said to me the other day, “In the future, there won’t even be software. It’ll all be AI.”

Sounds crazy right?  Or is it?

How AI replaced software (gpsd) for me today

It clicked for me today when I impulsively plugged an old USB GPS receiver into my laptop just to see if it worked.  I opened up minicom to see if it would get a fix, and quickly glazed over watching the stream of familiar but uninteresting log messages:

$GPGGA,,,,,,0,00,99.99,,,,,,*48
$GPGSA,A,1,,,,,,,,,,,,,99.99,99.99,99.99*30
$GPGLL,,,,,,V,N*64
$GPRMC,,V,,,,,,,,,,N*53
$GPVTG,,,,,,,,,N*30
$GPGGA,,,,,,0,00,99.99,,,,,,*48
$GPGSA,A,1,,,,,,,,,,,,,99.99,99.99,99.99*30
$GPGLL,,,,,,V,N*64
$GPRMC,,V,,,,,,,,,,N*53
$GPVTG,,,,,,,,,N*30
$GPGGA,,,,,,0,00,99.99,,,,,,*48
$GPGSA,A,1,,,,,,,,,,,,,99.99,99.99,99.99*30
$GPGSV,1,1,01,24,,,28*74
$GPGLL,,,,,,V,N*64
$GPRMC,,V,,,,,,,,,,N*53
$GPVTG,,,,,,,,,N*30
$GPGGA,,,,,,0,00,99.99,,,,,,*48
$GPGSA,A,1,,,,,,,,,,,,,99.99,99.99,99.99*30
$GPGLL,,,,,,V,N*64
$GPRMC,,V,,,,,,,,,,N*53
$GPVTG,,,,,,,,,N*30
$GPGGA,,,,,,0,00,99.99,,,,,,*48
$GPGSA,A,1,,,,,,,,,,,,,99.99,99.99,99.99*30

Those familiar with the GPS NMEA protocol can squint their eyes at this and can infer:

1. It doesn’t have a GPS fix yet ($GPGGA,,,,,,0,00,99.99,,,,,,*48 shows no latitude/longitude, just a bunch of empty comma-delimited fields)
2. It can see one satellite (the odd $GPGSV,1,1,01,24,,,28*74)

We have gpsd for this.  I don’t need to squint at this telemetry stream.  However a thought occurred to me: while I don’t have gpsd installed and ready to fire up on this GPS receiver, I do have a flexible AI agent I can throw at it.  It’s also quite possible that this GPS unit will never get a fix, so the event log of it finding specific satellites is interesting…it might be all that ever happens!

Claude wrote a bit of code to stream the contents of /dev/ttyACM0 to a log file on disk, and a Monitor process to provide updates back to the main agent loop.  It even detected the model of the GPS:

It then went from a stream of noisy events to something more useful.  It started to get events as it recognized more satellites:

This quickly was just noise.  I started asking it questions.  Will it ever get signal or is the placement just too bad?  How long do I need to wait on a cold start in a sub-optimal deployment environment like on my desk, below a monitor, indoors like I am trying right now?

While it monitored, I asked questions and it taught me things: GPS satellites move slowly across the sky, so flickering number of satellites in view is unlikely to be movement across the sky – it’s just weak signals.  This isn’t Starlink.

Then my friend’s point clicked.  I wanted less noise and I wanted to wait, so I let it keep trying and I asked it to rewrite it’s “software”:

I reprogrammed my “GPS status application” – Claude Code – on the fly.

The GPS eventually found a third satellite and after a few more minutes it got a fix as I had hoped for:

Could I write my own GPSd?  Sure I could.  But that’s not the point – the point is that using an AI agent in-situ means you don’t have to write the software up-front.  You don’t always need requirements and a plan like you did before the AI era.  Software can even be ephemeral like today’s exercise – none of the code it used even hit disk.

Here’s the full transcript if you’re curious to spelunk!

Further afield with OpenClaw

One of my OpenClaw experiments is to use it to track grocery spending and prices over time.  However I’m refusing to write any code right now.  All I’m doing right now is capturing data (photos of receipts) over time.  I know I’ll want to ask questions like “has the cost per pound of beef gone up over the past three months?”

My strategy for this is not to just optimize for the answers, but to see what questions the data set I’m reasonably able to collect can even answer.  Receipts aren’t perfect – will they prove useful?  I’m not going to code myself into a corner – instead I’m going to collect data and when we have enough, get the AI agent to build some analysis tools to try and answer my question.  Time will tell.

Challenging your own assumptions

Imagine the world of computing today had speedy NVRAM technologies ever competed with the volatile RAM + non-volatile “slow” disks, where applications have a distinction between running and not running.  Completely different mind set.  We need to apply that kind of thinking to AI!

We all know AI agents can write great code, but we’re still stuck in the old way of thinking that the software is the hard part.

Next time you think you need software, ask yourself: can an AI agent just do this for me?

A month or so back I overheard a conversation in my F3 Wheaton exercise group about a failed portable USB hard drive. The owner is professor Matthew Millner who has traveled as much as I have, has kids like me with priceless photos on that disk, and has ten years of PhD work and life on the disk.  Irreplaceable data.   I had to sit down and ask if I could help! If that was my data, I’d do anything to get it back!

Here’s how, over the span of 59 days, I was able to help.

I know, I know, people should make backups and/or use cloud storage.  But sometimes you don’t, and you just have to make the best of it.  He’s been hearing the lecture on backups for the past year since the disk died.  But as my grandfather-in-law would say, “If wishes were fishes, we’d all have a fry.”  Life is not a Douglas Adams novel: you can’t go back in time.  But you can try to do something about it.  The disk spins and shows a directory listing but none of the files are readable.  Some of it must be readable, right?

with all that padding around two spinning platters, how could it ever fail?

This is the story of how I helped get it back! I am not a hard drive recovery professional (and no, I did not stay at a Holiday Inn Express last night.) I have been taking computer hard drives apart since I was about ten years old.  In high school & college I went through a spree of dodgy 60GB hard drives which were fortunately in a RAID5 and I learned about freezer tricks, heat sinks+ice cubes, and the power of patient persistence. I’ve learned a few tricks since then that help: a sprinkle of microelectronics, and a dash of AI. Years of proximity to system administration taught me some tricks too; volume snapshots and copy-on-write filesystems and a sixth sense for failure modes.

The Impossible Job: a decade’s worth of PhD life on a failing 2TB drive

The disk is a LaCie 2TB portable USB hard drive.  Inside the package is a USB-SATA adapter with a standard 2.5″ SATA disk, a Seagate ST2000LM007 which has two spinning platters inside of it.

I plugged the USB enclosure in, got a directory listing, and the minute I tried to read anything it stopped.  That’s not a good sign!  I half expected to hear the traditional clicking noises of a drive which can’t seek at all and continually tries to re-seek, but I did not hear any so I was encouraged.  I moaned when I saw it was HFS+, but in retrospect, that was not a problem.  Every time I tried something, I had to unplug and re-plug the disk to get it to respond again.  Using the USB enclosure vs SATA directly didn’t change the behavior either.

I tried some fans to cool the disk to no effect, to the point of attaching some DS18B20 temperature sensors to each side of the disk to see if it was getting dangerously hot – it was not. But the graphs (in Home Assistant of course) proved useful later on.

First Contact: Why did dd keep dying at 40GB?

I suppose a hard drive expert can answer that question better than me!

My next step was to try cloning the disk with dd on linux.  Uncle Mintel left me a small cache of big 3.5″ old 2TB disks, and after some quick smartctl verification tests, I picked a couple that looked fit for the job.  I set up the disks on my workbench in the basement in an old PC that I use for GPU experiments, and started to try to clone the disk with dd.  Every time: it hit 40GB, the disk stopped, and the SATA link reset.  It needed a power cycle.  I clearly couldn’t afford to spend time re-reading the same blocks on the disk, so I needed a new plan.

ddrescue to the rescue!

ddrescue is dd but for recovery from a dying disk.  I’d never used it before, but it is exactly what I needed.

You point it at a source and destination disk, it tries to read, and keeps a map of every block it’s tried, what was copied successfully to the target disk, and what produced read errors.  You can run it iteratively, as many times as you want.

My disk was special: it never actually returned read errors; the disk simply performed so slowly that the host never saw a series of read errors – it just saw the disk time out and go away.  The useful part: you can restart ddrescue ad nauseum, at any arbitrary point on the disk, and it will focus on that area of the disk.  I quickly found that jumping ahead to 60GB that I could read the next 20-30GB cleanly.  However I’d eventually run into one of two problems: the disk would timeout and reset and need a power cycle, or a second failure mode emerged: the disk would read at 131kB/sec.  Mysterious?  Probably not – 131kB/sec is exactly 1000x slower than the “normal” read rate of the disk.  Presumably the disk was moving slowly because it was having to retry reads 1000x and using conservative error correction (which should be a pretty good indication of a physical failure, but I persisted!)

The problem with power cycling the disk all the time

I mean sure it’s annoying.  But what was really annoying to me was that it was in my basement; on top of which we have a newly-walking 1-year old baby.  I had to open the gate, go down; she’d want to come with so I’d grab her.  Pulling a cable out of a disk is a two handed operation, so I’d set her down on the basement floor, do the cable dance, pull out my SSH client on my phone, and tell the OS to refresh the SATA host.  Then grab the baby, go back upstairs, close the basement gate, set the baby down.  This got tiring quickly.

I’d learned a few years back that SATA host ports on Linux can be listed:

for host in /sys/class/scsi_host/host*; do
 h=$(basename $host)
 ata=$(readlink $host | grep -oP 'ata\d+')
 if [ -e $host/device/target* ]; then
  blk=$(ls $host/device/target*/*/block/)
  model=$(cat $host/device/target*/*/model)
 else
  blk="<none>"
  model="<none>"
 fi
 echo "$ata -> $h -> $blk -> $model"
done

For example, my file server:

root@minor:~# ./list-sata-devices 
ata1 -> host0 -> sda -> Samsung SSD 850 
ata2 -> host1 -> sdb -> Samsung SSD 850 
ata3 -> host2 -> <none> -> <none>
ata4 -> host3 -> <none> -> <none>
ata5 -> host4 -> sdc -> Samsung SSD 850 
ata6 -> host5 -> sdd -> Samsung SSD 850

You can then tell the scsi host to refresh a port that you just hotplugged:

echo "- - -" > /sys/class/scsi_host/host4/scan

I’ve yet to find a consumer grade SATA host which doesn’t tolerate hotplugging.

I had that – but I needed an automated way to power cycle the disk and save both me and the 1-year old the trips to the basement.

ESPHome ESP8266 + relay on the 5V Rail

Because hey, why not?  I have north of 30 esp8266 MCUs that I’ve built stuff on, all plugged into Home Assistant.  I borrowed a relay node that I use to power cycle DFM radiosondes when on the bench and wired it up to the 5V rail on a SATA power adapter (I left 12V unplugged given that little disks don’t even use the 12V rail from what I’ve read.)  A HASS auth token and a bit of curl later, and I had a script that did an end-to-end power cycle of the disk:

• Power off the disk and wait a few seconds.
• Refresh the SCSI (SATA) port to make sure the kernel sees the disk is gone
• Power on the disk and wait a few seconds
• Refresh the SCSI port again.
• If the disk didn’t come back, sleep for 60 seconds and try once more, and if it still doesn’t work, print a big error on the screen and wait for user input.

This setup was the key.  I left the basement one day, and didn’t touch the host for an entire month afterwards as I slowly worked on it.

Timeouts as a Strategy: Skipping the Tar Pits

F3 AO pun not intended

Now that I had a working power cycle procedure, I could try reading the entire disk in segments.  I wrote a simple wrapper which ran ddrescue at 10GB increments, wrapped in a bash “timeout 120” which meant that if I hit one of those 131kB/sec slow spots, or the disk timed out and needed a power cycle, that ddrescue would exit out.  The loop turned into:

• For x in (10GB segments up to 2000GB)
  • timeout 120 ddrescue read as much as you can
  • Test the disk. Is it offline?  If so, power cycle it with the procedure above.

This was gold.  In a couple of days, I had recovered 724GB of the disk’s total 2000GB area.

Making it Visible: Writing ddrescue log visualizers & HA graphs

The log file is a long machine-readable text file not meant for humans.  It has one block range per line, so on every ddrescue run it got longer.  The ddrescuelog CLI tool didn’t work for me for some reason.  I threw this one at Claude Code which wrote a CLI and HTML visualizer:

I already had the relay switch visible on Home Assistant, which yielded a history graph showing if the disk was powered on or off and the corresponding temperature sensors below it, so you can glance at what was happening autonomously while you were sleeping:

I went further – as regular readers may know, I am obsessed with getting every bit of my data into Home Assistant…read on!

Zooming In: From 10G to 1G to 100M resolution passes

I revised my wrapper script to then do 1G segment passes, with a correspondingly shorter timeout, and eventually I went on to 100M segments.  Smaller segments took much longer; a lot of the remaining data surface area was reading at the slow 131kB/sec rate.  Every now and again, my power cycle controller would hang – I’ve found that a Home Assistant server hang has weird knock-on effects for my esp8266 nodes.  OneWire temperature sensors start returning 80C like parasitic power problems and my relay GPIO would stop toggling, and I’d have to power cycle the node! Still unresolved.

At the completion of the 100M chunk pass, I had a lot more of the data, but I estimated that if I let it run on every block, it would take another 45 days to read it all at 131kB/sec – and that’s not counting power cycles (which became so numerous I stopped paying attention and just kept hoping that the next power cycle would not be the disk’s last.)

Tangent: proof of life (and an excuse to use another hard drive)

I needed to see if I was getting anywhere. I took a few hours’ pause from ddrescue and cloned my recovery target disk over to a second spare online 2TB disk. Pro tip: hotplug disk names like /dev/sdd are not stable across reboots! You knew that. Check your serial numbers or disk device IDs; fortunately in this entire process I never rebooted once (nor did the mains so much as flicker) until the process was complete.

Once my dd to the spare disk was complete, I wanted to mount the volume and run fsck_hfs but I also wanted to snapshot and work with a copy-on-write (CoW) image of the disk. The underlying partition was 100% the size of the physical disk, so the CoW store needed to live elsewhere. My OS disk is a mostly empty 1TB lvm volume which unhelpfully is 100% allocated and I didn’t want to go offline just to lvresize.

I asked Claude.  It’s a pretty good sysadmin!  I created my CoW store as a file on the OS disk, made a loopback device, and used the dm-snapshot method.  Worked perfectly.

I ran fsck and mounted the volume.  I clicked around.  My jaw hit the floor. Even with many GB still to recover, I had tons of photos and sent the owner some proof-of-life photos!  Needless to say, he was excited!

The Breakthrough: Reading the HFS+ allocation file (aka bitmap)

Big thank you to Matthijs Kooijman for their article Recovering data from a failing hard disk with HFS+

This article described a lot of the same challenges I had, and coincidentally also the same filesystem.  The power cycling tricks described there never worked for me, but the HFS+ pointers absolutely did: HFS+ maintains an allocation file which gives a map of which blocks on the disk were in use.  Cross-reference this with the ddrescue log file, and I was in business. I knew exactly which blocks to focus on recovering!

I prompted Claude Code to write a hfsplus_priority.py script to generate 16,000 arbitrarily-sized ddrescue commands for all the remaining blocks, and also publish this to MQTT.  The generated bash script would run for days; the MQTT publish I ran hourly to make me my pretty graph.  This was fun – I could post to the F3 Slack on occasion for people to cheer for the little hard disk that could:

This took another week or so.  At which point my graph levelled out at 40G to go.  Only a few segments in the ~16,000 had failed to read and weighed mere megabytes each – what did I miss?  The ddrecover master process said it was complete!

Trust, but Verify: The day Claude admitted to “helpfully” skipping blocks under 1MB

I teach classes on Claude Code.  I teach people to find ways to verify what it comes up with.  I’m well beyond Spicy Autocomplete; I’m often the FSD driver asleep at the wheel except instead of sleeping, I am parenting.  I should have expected this. In the hfsplus_priority.py script, Claude Code helpfully added a --min-size flag which defaulted to skipping blocks smaller than 1MB.  Claude’s context at this point in the project was huge, and full of references to iteratively trying to recover the largest ranges of data I could assuming that the disk would eventually crash.  It didn’t take long to find the problem (I asked Claude!) and I was on my way. This last recovery sweep added another few days and thousands more segments to cover, most of which were tiny, but added up to 40GB.

Almost there…

The “bytes to recover” graph in Home Assistant marched on to nearly zero.  A few megabytes were truly unreadable no matter how many times I tried, and every time I tried the disk needed a power cycle.  I was done.  I’d recovered all I needed to.

Integrity audit & reporting

I did the same dm-snapshot CoW setup, this time for the primary recovery disk.  Browsing around the volume was like living in a dream: it all worked!  But did it?  There are 600,000 files on the volume.  Let’s be systematic.

I prompted Claude Code to work up an auditor script which attempted to open every file and see if its file type and detected mime type (using the file command mime-info magic database) matched, and then I extended it to actually attempt to open every single file that it could using a Python library to see if the input data was sensible.  Reality hit: loads of files were unreadable, about 1/6 of the total count of files.  A bit of heuristic analysis showed that most of the unreadable files were auto-generated iPhoto thumbnails.

That said: a darn good result given that the data owner had already written it all off.

I took counts of file types, broken down by recovered vs lost, and had Claude Code write a little user-friendly report summary.  I packaged the 800G of data (now some of it gibberish, left in-place) and copied it to the user’s new scratch disk, a solid-state disk, which he said was bound for iCloud.  A good move for important data!  (I have since confirmed the data has made it to iCloud! Long live the data!)

Fin. Done.  I shut down the recovery computer.  Uptime: 59 days elapsed from start to finish.

Lessons from an Impossible Project

“No problem can withstand the assault of sustained thinking.” – Voltaire
“Nothing is impossible, except skiing through a revolving door.” – Woody Allen

the “rig” with the hard drive under examination in the upper-right on a piece of wood

A bit of AI and experience sure helps.

This is one of the joys of being self-employed: the idea that my effort, my drive, is all mine.  I volunteered for a seemingly impossible job, one which I would not normally pick up.  I said yes, expecting to lose and get nothing in return. When the going got tough, I still said “what else” and used a lot of tools.  AI was helpful, not just for technical coding, but also as a consult.  How many platters does this hard drive have?  Why is it pulling data so slowly?  Why is it timing out from the SATA bus?  How do I power cycle a disk on Linux?  Is there anything  I can do about the speed?  Can I tell the firmware to stop retrying so hard?  What’s the syntax for smartctl?  I can’t count on two hands the number of disparate chat threads I’ve made across Claude and ChatGPT and Gemini about this.  I’ve made so many on Claude that it now has a memory of the model of disk, and every now and again guesses that I’m talking about this disk when I ask an unrelated linux block device question!

Next World Backup Day, I’ve got a story to tell.

If you’re like me and constantly running up your Claude subscription usage to its limits, you want a graph. Anthropic has two rate limit windows; one which is five hours long and one which is a week. Go beyond either and you need to put in a credit card for extra usage, upgrade to Max, or – wait! What kind of AI charged engineer wants to wait?

I do more than half of my usage on my mobile phone, so I wanted the best and easiest graph platform a mobile user can get: Home Assistant.

Check out the open sourced code: https://github.com/trickv/hass-claude-usage

Installation with HACS is easy:

1. 1. Add the repository as a custom repository in HACS
   2. Restart Home Assistant
   3. Install “Claude Usage”
   4. Restart Home Assistant
   5. Go to Settings → Devices & Services → Add Integration → “Claude Usage”

Once it starts collecting data, make a History Graph with the data points and the window you find this useful for; I usually look at the 24 hour window. See the README for an example dashboard.

Let me know if you find this useful or are managing your usage in other ways?

How I built it – iteratively

Building this little tool was certainly different to how I would have approached this in the pre-AI past.  I would have defined not just the end goal but also the methods clearly and worked straight towards it, optimizing to spend as little time coding as possible.  Given that this is a hobby project, I probably never would have finished it!  The API was elusive at first and I couldn’t find examples for how to use it.  But the AI-enabled engineer approach lends us to an iterative approach, where code is cheap, and making something that works is important to justify spending more time on.  My iterations looked something like this:

1. First version which used the HA API to create sensors, and used Claude Code CLI in a tmux session to get usage data. It would run Claude Code, wait a few seconds, type /usage, and grep the output.  It worked, albeit running claude code periodically contributed to usage!  But it gave me a graph, and it was useful.
2. Refactored to run Claude Code and keep it open in a persistent tmux
   1. Somewhere in here I learned about Anthropic’s rate limits on the usage endpoints which block you for 24 hours when you hit them too much!
3. Learned how to use the APIs directly to fetch usage status, but this approach had to steal Claude Code’s session token (which expires every few hours).
4. Moved from HA API to MQTT integration to get better data into Home Assistant. This was a surprisingly easy refactor for Sonnet 4.5!
5. Refactored to standalone OAuth (first time I used Opus 4.5 on this code)
6. Now refactored into this Home Assistant custom integration, installable by HACS

How are you managing usage?

I’m curious to hear how others on low usage plans like mine (Pro) are managing their usage.  Let me know if  you find this useful!

The answer to life, the universe, and everything.

(But what, you say, is the question?!)

I’ve done a lot in the past 41 years, but for my next lap around the sun, I have added a special goal: become more whimsical. If that’s even possible.

Douglas Adams left us a gold mine of silly stories, many of which I’ve read and this coming year I will re-read. I might even turn it into a weekly newsletter for 42-year olds. Let me know if that idea appeals to you.

In the meantime –

Everything will be fine. I’m just a year older. In fact, I’m just a day older than I was yesterday.

~ Trick, 2025-12-13