Data in space: no one can you hear you scream

A 20-stone Panamanian DBA who fancied himself as a bit of an event data management specialist once cornered me at a developer event to talk my ears off while I was trying to beat my own personal hot dog and beer consumption record. My data-centric pal (let’s call him Pancho) beat my eardrums up about architectural design for real world retail applications that experience wild spikes in terms of data throughput once in full deployment.

I can talk about Pancho comfortably now as I ran into him at Sybase’s TechWave symposium and while it’s being held in Washington DC today, I am safely ensconced in Surrey.

So in the fullness of time, I do think back to that bizarre conversation we had in the depths of a Caesar’s Palace conference hall. It focused on planning for events inside an RDBMS for the retail industry and dealing with unknown waves of data caused by unpredictable consumer behaviour. Actually, it was interesting after all.

I was reminded of my Panamanian pal when I saw the release of a new book this week by O’Reilly called Beautiful Data - The Stories Behind Elegant Data Solutions.

This book attempts to explain how “prominent” data designers, managers, and handlers solve some of the most difficult problems when working with data. In doing so the authors look at areas like the opportunities and challenges involved in working with the vast number of datasets made available by the web.

So nice book – but the connection to Pancho the Panamanian and event data spikes and my space related blog headline…?

The book also digs into the challenges of designing a data processing system that works within the constraints of space travel.

So you think a retail system is hard to plan and provision for? Try a space shuttle, or even a Virgin Galactic space plane if you prefer. If you get that data forecasting wrong, no one can hear you scream. It makes you wonder how they did it in the 60s doesn't it?

NASA’s own Intelligent Spacecraft Interface Systems lab works with data inside an integrated tool using an empirical database which computer science personnel are using to develop and validate computational models of human behaviour and human performance to help predict things like crew reaction time.

But still, there are a lot of unknowns to factor in right? I’m no expert on space travel or space technology, but I saw this book launch, thought about the subject, remembered Pancho the mad Panamanian and well, just couldn’t help ramble on.