More Data, More Problems

Without a doubt, big data has transformed how companies optimize profits and reduce loss.  Retailers have become empowered with tracking customer buying patterns to build targeted ads and offers, increasing sales by focusing on purchasing needs.  Airlines are able to dynamically calculate flight pricing based on demand for certain routes, maximizing profits while limiting the number of empty seats.  And through sensors and other machine-generated data, predictive analytics can identify system malfunctions prior to a manufacturing system outage.  

It’s hard to imagine a time when usable insights weren’t so easy to compile.  Starting with a simple goal, data scientists are able to define data needs, architect indexing and storage on an open source or proprietary system, and build models to identify trends & produce answers.  Data in, insights out.

However, this strategy doesn’t always work in cybersecurity.  As many security teams aren’t sure how threats will evolve, many pick a strategy to collect “all the things” in hopes of making sense of it later.   That “later” part could be anything from detecting threats that didn’t previously have a known signature, to having forensic data available for incident investigation.  

Without knowing exactly what they’re looking for, log data of various types and formats are forced into long term storage.  Many have dual-purposed their SIEM, crossing the streams when it comes to objectives (compliance reporting vs. threat remediation).  And in extreme cases, separate “data lakes” are developed for collection and retention with little forethought as to how the data can be indexed and searched.

Unfortunately, the industry has lost focus on what problems it's trying to solve.  By starting with the proposition that “threats will always 0wn us” isn’t a logical position.  Designing a security program around specific objectives, then collecting the right data to answer the right questions at the right time will considerably reduce the amount of damage that a threat can do.

In cybersecurity, it’s not the size of your data, but how you use it.