Thinking Different, Thinking Slowly: LLMs on a PowerPC Mac

There is something incredibly satisfying about breathing new life into old hardware. Vintage computing is one of my favorite hobbies. The challenge of coaxing modern software onto systems designed decades ago is a puzzle I cannot resist. I have been diving into the world of large language models (LLMs), and a question began to gnaw at me: could I bring the cutting-edge of AI to the nostalgic glow of my trusty 2005 PowerBook G4? Armed with a 1.5GHz processor, a full gigabyte of RAM, and a limiting 32-bit address space, I embarked on an experiment that actually yielded results. I have successfully managed to achieve LLM inference on this classic piece of Apple history, proving that even yesteryear's hardware can have a taste of tomorrow's AI.

PowerBook G4 running TinyStories 110M Llama2 LLM inference

I started by reviewing the llama2.c project from Andrej Karpathy. This brilliant project implements Llama2 LLM inference with just a single file of vanilla C. No accelerators here. Performance is traded for simplicity which makes it easy to understand how inference is carried out.

I forked the core implementation to a project that I have titled ullm. The core algorithm remains the same, but I spent time improving a few aspects of the code so that it would stand up to abuse a little better.

12V: Ryobi 40V Lithium Battery Charger

I have been impressed with Ryobi's 40V battery packs lately. They are fairly energy dense and compatible with a wicked 1800W inverter/power station that I have. Internally they are a 10S lithium pack with some number of cells in parallel depending on the pack capacity. They use the safer flame proof cells and have a sophisticated BMS built-in so I feel pretty comfortable messing with them. They can also be obtained at steep discounts if you are willing to tolerate used packs from eBay. In my experience, they test out fine.

I came to the realization that it would be handy to be able to charge these batteries in the car. I drive electric vehicles and in the event of a power failure, these packs could easily be charged up numerous times. Ryobi doesn't make a 12V-compatible charger for this line of batteries so I took matters into my own hands.

12V Compatible Ryobi 40V Battery Charger

This charger uses an adjustable constant-current/constant-voltage (CC-CV) power supply that steps the 13.8-14.4V that my car provides up to the 42V needed to charge the batteries. I tore apart the OEM chargers that Ryobi provides and found that the weakest link is a blocking diode rated for 3A. The OEM charge adapter ordinarily operates at 1.5A from a wall-powered CC-CV supply. I configured my DC-DC converters to operate at a peak voltage of 42V with 2.75A of current, just shy of the limit of the blocking diode. So far, all has been working well. There is a slight increase in temperature of the case plastics of the charger, but nothing outside of my own personal comfort zone.

Ryobi 40V Batteries Charging in the Car :]

The result is that I can now easily charge my batteries in the car. I decided not to use a 12VDC to 120VAC inverter to avoid any efficiency penalty and have measured this design to be around 90% efficient. In this configuration, the batteries are charging at 100W+ which is pretty impressive. In order to mitigate thermal losses, a small 12V fan is employed to move heat away from the converter.

Ryobi 40V Car Charger Assembly

The assembly consists of a few 3D printed components. I designed a case to accommodate the DC-DC converter, some panel mount XT-60 connectors and the cooling fan. I also designed a backshell for the XT60 connectors to make them easier to manipulate. They turned out great.

XT60 Backshell Custom Design

I used a cheap rubber strain relief to make the cables more robust. The end result is a convenient device that should come in handy for power outages, camping trips or any other time that it makes sense to keep batteries topped off in the field.

Finished Assembly

I sacrificed the wall power supply for the connector to use for the charger. All is not lost though. I fitted the opposite gender XT60 connector and an identical backshell so that the charger can still be used in the factory original configuration or for charging from a car.

I hope you enjoyed this short blog post and small project of mine!

QRTape | Audio Playback from Paper Tape with Computer Vision

Here is a project that I have been tossing around in my head for at least a year or two now. Given advances in audio compression algorithms and computer vision: could reasonably high-quality audio be stored on a paper tape?

This is a fascinating concept to me. When considering the complete history of storage media from early cylindrical engraved records through to magnetic tape it is evident that achieving high quality and reliable data storage is both challenging and expensive. When specifically considering magnetic tape, the challenges surrounding complex mechanical tape transports and sensitive electronics highlight some of the difficulties that engineers have faced in the past.

QRTape

In this blog I demonstrate a system that exploits modern computer vision and audio compression to replace the complex mechanical tape transports of the past. In fact, my tape transport is made entirely from paper and cardboard (excluding some electronics, of course). I have decided to call this system QRTape.

QRTape Player

Under the QRTape system, data is encoded by a series of QR codes that are printed on a continuous strip of paper. This strip of paper is fed from one spool through a crude tape transport, past a webcam and onto a take-up spool. The paper is advanced by a small stepper motor driven by a cheap Arduino. The rest is pure software magic.

nrfnet: Streaming Video over nRF24L01

A couple of days ago I published a blog discussing how I used NRF24L01 radios to implement a point-to-point network between two Raspberry Pi computers. I implemented this as a virtual network device and sent packets between the radios.

Since then, I have made numerous improvements to the software and more than tripled the throughput from ~90kbps to nearly 300kbps. These improvements were through a variety of changes that I will cover in this blog post.

Streaming video from one headless Raspberry Pi to another

Thanks to the higher throughput, I was able to implement streaming video using the h264 HEVC video codec and monaural audio using the Opus codec at 32kbps. The result is great, especially when considering the link.

Continue reading to learn more!

nerfnet: Wireless Networking over nRF24L01 2.4GHz Radios

I recently picked up a set of nRF 2.4GHz radio transceivers. These are low-cost radios with a SPI interface that allow exchanging 32 byte packets across a radio link that can run at up to 2MBit on-air data rates. They are popular among hobbyists who want to introduce wireless to their Arduino-flavored projects. I was able to buy ten of these radios with trace antennas for just $11 and three more with SMA-connected antennas for $18.

NRF24L01 Radios

My first inclination is to try something a bit more extreme with this hardware. There is a GitHub project named RF24Audio that allows transmitting audio data over these radios. I wondered if video could be possible and started brainstorming about how a video transport over this link would look. The further I got into the specifics of streaming video the more convinced I was that an abstract link that could carry any form of data would be more fun.

This led me to build nerfnet: a simple application that allows sending network frames over NRF24L01 radios. This is implemented by exploiting the TUN/TAP virtual network device API under Linux on a Raspberry Pi. The code is available on GitHub for you to review and use.

I was able to demonstrate nearly 90kBit throughput as measured by iperf. I suspect this is the first time that iperf has been used to characterize a link composed of these radios.

?

1 2 3 4 5 6 7 8

andrew@andrew-pi:~/Projects/nerfnet $ iperf -c 192.168.10.2 ------------------------------------------------------------ Client connecting to 192.168.10.2, TCP port 5001 TCP window size: 43.8 KByte (default) ------------------------------------------------------------ [  3] local 192.168.10.1 port 34490 connected with 192.168.10.2 port 5001 [ ID] Interval       Transfer     Bandwidth [  3]  0.0-10.1 sec   110 KBytes  89.4 Kbits/sec

Continue reading or watch the video to learn more about how I pulled this off.

Plaid Model S Excitement

If you have been following my blog for a while, you probably know that I am an unabashed Tesla fan. My heart lies with the Model S, the only car that I have ever bought twice.

The recent Laguna Seca record-setting lap has me excited. One thing that stood to me was the e-stop stuck to the dash. I wonder what it does. Does it cut the main battery pack contactor? Wouldn't it be too late by the time it needs to be disengaged? Does it dump fire suppressant over the vehicle? Well, I might be too far out of the loop to ever know what function it performs but decided to re-create the look for fun.

The Resistor Network Plaid Model S :]

The total cost was $15 for a pair of dubious e-stop switches and a couple of small squares of 3M VHB. I put on my longboarding attire and snapped a few photos with my new Canon R5. After a little futzing about in Adobe Lightroom, I ended up with this.

My blog has been a little light on projects lately. Work definitely keeps me busy. It is fun to head out and be creative every once in a while though. This was taken near The Computer History Museum in Mountain View, California. There is a Supercharger nearby.

I hope you enjoy my fun and congratulations to Sebastian Vittel on the impressive lap!

Amateur Radio from 2900ft. on Mount Diablo | 146.520MHz FM Simplex

Over the past weekend I took a solo, socially-distanced road trip to Mount Diablo to try out my Amateur Radio station. The results far exceeded my expectations and I ended up making more than 25 contacts total.

View from Mount Diablo with Antenna on Trunk

I decided to put together a video about my adventure. It turned out to be a nice day and a good way to get out of the house. The Amateur Radio Community is incredibly friendly which made for many smiles and fun contacts.

I decided to QSL as many of these contacts as possible. I hope to get some interest cards back from the various operators that I was able to reach.

First QSL Card Design: Lick Observatory

Thanks for reading and I hope you enjoyed the video :]

datvideo: Storing Video on Digital Audio Tape (DAT)

About a year ago I added a Digital Audio Tape (DAT) deck to my home theater system. I was turned onto the format by popular Youtuber Techmoan and a great video that he produced: Digital Audio Tape: The one DAT got away. I had wanted to add magnetic tape to my home theater for some time and this highly unique format seemed like a perfect choice.

Big Buck Bunny from DAT Tape

Shortly after purchasing the deck I realized that I could likely store any arbitrary binary data on the tape with the S/PDIF input/output. It would be really cool to merge some modern high-compression video codec with this antiquated format. Over this weekend I decided to do just that.


I wrote a small tool called datvideo that allows storing arbitrary binary data on the tape. This tool is used to grab raw audio binary data from a sound card, search for frames of binary data, decode them and emit them into another file. This can be assembled into a pipeline to feed video data into a player such as mplayer. How cool is that?

Sony DTC-690, below Marantz Blu-Ray, AV Receiver and New-Old-Stock DAT Tapes

In this article, I will walk you through how I pulled this off.