Adventures in Tech

In recent years there has been an explosion of data all around us. The data comes in from a variety of sources, such as financial real-time systems, cell phone networks, sensor networks--RFID and IoT, and GPS. Commensurate with this dramatic increase in data, is a corresponding unquenchable thirst for analysis and insights. The natural question arises: how do we build systems that process and makes sense of this vast amount of data, in as close to real-time as possible? What patterns of software and systems should we look at? Michael Stonebraker of database fame et al. offer some advice on what to consider in their paper: " The 8 requirements of real-time stream processing " published a decade ago. In the paper, the authors list eight guiding principles that high-volume low-latency systems should follow to be able to process vast amounts of data in near real-time. First, the systems have to keep the data moving, and do straight-through processing with minimal to no writ...

In the Pacific Northwest, a lot of people use their trucks as everyday commute vehicles, which makes sense, since the climate is wet, and the terrain hilly, and in wet and cold conditions people feel safer in their four wheel drive vehicles. Some of these trucks are the heavy duty ones, with big diesel engines, and lately I have noticed at work a couple idling sans driver for at least 5 minutes. It got me curious about why would you need to idle a diesel engine especially since modern gasoline engines do not require idling before driving off and putting load on the engine. A web search helped piece the answer to this puzzle. Diesels operate differently than gasoline engines. Instead of relying on spark plugs to light up the air and fuel mixture inside of the engine cylinders, diesels rely on high compression ratios that cause the air and fuel mixture inside of the cylinders to ignite.  Because of the high compression ratios, diesel engines are typically bulkier and more stu...