Welcome!

PHP Authors: Liz McMillan, Carmen Gonzalez, Hovhannes Avoyan, Lori MacVittie, Trevor Parsons

Related Topics: @CloudExpo, @DXWorldExpo, @ThingsExpo

@CloudExpo: Blog Post

The #IoT and #Analytics | @ThingsExpo #BigData #BI #AI #DX #MachineLearning

The Internet of Things promises to change everything by enabling “smart” environments and smart products

The Internet of Things (IoT) and Analytics at The Edge

The Internet of Things (IoT) promises to change everything by enabling “smart” environments (homes, cities, hospitals, schools, stores, etc.) and smart products (cars, trucks, airplanes, trains, wind turbines, lawnmowers, etc.). I recently wrote about the importance of moving beyond “connected” to “smart” in a blog titled “Internet of Things: Connected Does Not Equal Smart”. The article discusses the importance of moving beyond just collecting the data, to transitioning to leveraging this new wealth of IoT data to improve the decisions that these smart environments and products need to make: to help these environments and products to self-monitor, self-diagnose and eventually, self-direct.

But one of the key concepts in enabling this transition from connected to smart is the ability to perform “analytics at the edge.” Shawn Rogers, Chief Research Officer at Dell Statistica, had the following quote in an article in Information Management titled “Will the Citizen Data Scientist Inherit the World?”:

“Organizations are fast coming to the realization that IoT implementations are only going to become more vast and more pervasive, and that as that happens, the traditional analytic model of pulling all data in to a centralized source such as a data warehouse or analytic sandbox is going to make less and less sense.

So, most of the conversations I’m having around IoT analytics today revolve around looking at how companies can flip that model on its head and figure out ways to push the analytics out to the edge. If you can run analytics at the edge, you not only can eliminate the time, bandwidth and expense required to transport the data, but you make it possible to take immediate action in response to the insight. You speed up and simplify the analytic process in a way that’s never been done before.”

So I asked Shawn and his boss John Thompson, General Manager of Advanced Analytics at Dell, to help me understand what exactly they mean by “analytics at the edge.” It really boils down to these questions:

  • Are we really developing analytics at the edge?
  • If not, then what sorts of analytics are we performing at the edge?
  • Where are the analytic models actually being built?
  • And finally, what the heck does “at the edge” really mean?
  • So let’s actually start with that last question: What does “at the edge” really mean?

Question #1: What Is “At The Edge”?
“At the edge” refers to the multitude of devices or sensors that are scattered across any network or embedded throughout a product (car, jet engine, CT Scan) that is generating data about the operations and performance of that specific device or sensor.

For example, the current Airbus A350 model has close to 6,000 sensors and generates 2.5 Tb of data per day, while an even newer model – expected to be available in 2020 – will capture more than triple that amount! It is becoming more and more common for everyday common products to have hundreds if not thousands of embedded sensors that are generating readings every couple of seconds on the operations and performance of that particular product (see Figure 1).

Figure 1: Sensors at the Edge

But collecting these huge and real-time volumes of data doesn’t do anything to directly create business advantage. It is what you do with that data that drives the business value, which brings us to…

Question #2: Are We Really Developing Analytics “At The Edge”?
Are we really “performing analytics” (collecting the data, storing the data, preparing the data, running analytic algorithms, validating the analytic goodness of fit and then acting on the results) at the edges, or are we just “executing the analytic models” at the edges? It’s one thing to “execute the analytic models” (e.g., scores, rules, recommendations) at the edges, but something entirely different to actually “perform analytics” at the edges.

Per Shawn and John, “We can deliver analytic models to any end point. We can execute the analytic models in any environment – large or small. We can execute all the steps in performing analytics in a wide range of environments, but there are limits at the edge. The limits are on the robustness of the environment (i.e. cannot deliver an executable to an environment that does not have the memory or processing power to store it or execute it. We cannot change the laws of physics…;-).)”

Question #3: What Sorts Of Analytics Are We Performing At The Edge?
In our airplane example with 6,000 sensors on the plane generating over 2.5 Tb of data per day, how are we performing the analytics at the end?

Per John and Shawn, if the jet engine has a place to house a Java Virtual Machine (JVM) and an analytic model (i.e., lightweight rules based model), then we can execute the model on the engine itself. If the model streams the data to a network, we can execute the analytic model on a gateway, or intermediate server (see Figure 2).

Figure 2: Executing Analytic Models at The Edge

Think of the network as having concentric rings. Each ring can have many servers. Each server can do either – either executing an analytic model or building the analytic models. Now think of many network networks with concentric rings that interlock at various intersections. Analytics can be at any or all levels including at the core, in a data center or in the cloud.

Per Shawn, “By working in tandem with Dell Boomi, we’ve given users the ability to deploy JVM’s with the analytic models on any edge device or gateway anywhere on the network or device. This edge scoring capability enables organizations to address nearly any IoT analytics use case by executing the analytic models at the edge of the network where data is being created.”

Question #4: Where Are The Analytic Models Actually Being Built?
Okay, so we “execute” the pre-built modes at the edge, but we actually build (test, refine, test, refine) the analytic models by bringing the detailed sensor data back to a central data and analytics environment (a.k.a. the Data Lake). Figure 3, courtesy of Joel Dodd of Pivotal, shows the data flow and the supporting analytics execution.

Figure 3: “At the Edge” Analytic Model Execution

Final point, even if you are doing all the sensor/IoT analysis at the edges, you are likely still going to want to bring the raw IoT data back into the data lake for more extensive analysis in order to house the detailed IoT history. For example, we have major economic cycles every 4 to 7 years. You might want to quantify the impact of these economic changes on your network demand and performance. That would eventually require 8 to 14 years of data. And that’s why you are going to want a data lake as the foundation of the transition from a “connected” IoT world to a “smart” IoT world.

The post The Internet of Things (IoT) and Analytics at The Edge appeared first on InFocus.

Read the original blog entry...

More Stories By William Schmarzo

Bill Schmarzo, author of “Big Data: Understanding How Data Powers Big Business” and “Big Data MBA: Driving Business Strategies with Data Science”, is responsible for setting strategy and defining the Big Data service offerings for Dell EMC’s Big Data Practice.

As a CTO within Dell EMC’s 2,000+ person consulting organization, he works with organizations to identify where and how to start their big data journeys. He’s written white papers, is an avid blogger and is a frequent speaker on the use of Big Data and data science to power an organization’s key business initiatives. He is a University of San Francisco School of Management (SOM) Executive Fellow where he teaches the “Big Data MBA” course. Bill also just completed a research paper on “Determining The Economic Value of Data”. Onalytica recently ranked Bill as #4 Big Data Influencer worldwide.

Bill has over three decades of experience in data warehousing, BI and analytics. Bill authored the Vision Workshop methodology that links an organization’s strategic business initiatives with their supporting data and analytic requirements. Bill serves on the City of San Jose’s Technology Innovation Board, and on the faculties of The Data Warehouse Institute and Strata.

Previously, Bill was vice president of Analytics at Yahoo where he was responsible for the development of Yahoo’s Advertiser and Website analytics products, including the delivery of “actionable insights” through a holistic user experience. Before that, Bill oversaw the Analytic Applications business unit at Business Objects, including the development, marketing and sales of their industry-defining analytic applications.

Bill holds a Masters Business Administration from University of Iowa and a Bachelor of Science degree in Mathematics, Computer Science and Business Administration from Coe College.

@ThingsExpo Stories
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
"MobiDev is a software development company and we do complex, custom software development for everybody from entrepreneurs to large enterprises," explained Alan Winters, U.S. Head of Business Development at MobiDev, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
22nd International Cloud Expo, taking place June 5-7, 2018, at the Javits Center in New York City, NY, and co-located with the 1st DXWorld Expo will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud ...
@DevOpsSummit at Cloud Expo, taking place November 12-13 in New York City, NY, is co-located with 22nd international CloudEXPO | first international DXWorldEXPO and will feature technical sessions from a rock star conference faculty and the leading industry players in the world.
JETRO showcased Japan Digital Transformation Pavilion at SYS-CON's 21st International Cloud Expo® at the Santa Clara Convention Center in Santa Clara, CA. The Japan External Trade Organization (JETRO) is a non-profit organization that provides business support services to companies expanding to Japan. With the support of JETRO's dedicated staff, clients can incorporate their business; receive visa, immigration, and HR support; find dedicated office space; identify local government subsidies; get...
Dion Hinchcliffe is an internationally recognized digital expert, bestselling book author, frequent keynote speaker, analyst, futurist, and transformation expert based in Washington, DC. He is currently Chief Strategy Officer at the industry-leading digital strategy and online community solutions firm, 7Summits.
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
In past @ThingsExpo presentations, Joseph di Paolantonio has explored how various Internet of Things (IoT) and data management and analytics (DMA) solution spaces will come together as sensor analytics ecosystems. This year, in his session at @ThingsExpo, Joseph di Paolantonio from DataArchon, added the numerous Transportation areas, from autonomous vehicles to “Uber for containers.” While IoT data in any one area of Transportation will have a huge impact in that area, combining sensor analytic...
Bill Schmarzo, author of "Big Data: Understanding How Data Powers Big Business" and "Big Data MBA: Driving Business Strategies with Data Science," is responsible for setting the strategy and defining the Big Data service offerings and capabilities for EMC Global Services Big Data Practice. As the CTO for the Big Data Practice, he is responsible for working with organizations to help them identify where and how to start their big data journeys. He's written several white papers, is an avid blogge...
Charles Araujo is an industry analyst, internationally recognized authority on the Digital Enterprise and author of The Quantum Age of IT: Why Everything You Know About IT is About to Change. As Principal Analyst with Intellyx, he writes, speaks and advises organizations on how to navigate through this time of disruption. He is also the founder of The Institute for Digital Transformation and a sought after keynote speaker. He has been a regular contributor to both InformationWeek and CIO Insight...
Michael Maximilien, better known as max or Dr. Max, is a computer scientist with IBM. At IBM Research Triangle Park, he was a principal engineer for the worldwide industry point-of-sale standard: JavaPOS. At IBM Research, some highlights include pioneering research on semantic Web services, mashups, and cloud computing, and platform-as-a-service. He joined the IBM Cloud Labs in 2014 and works closely with Pivotal Inc., to help make the Cloud Found the best PaaS.
It is of utmost importance for the future success of WebRTC to ensure that interoperability is operational between web browsers and any WebRTC-compliant client. To be guaranteed as operational and effective, interoperability must be tested extensively by establishing WebRTC data and media connections between different web browsers running on different devices and operating systems. In his session at WebRTC Summit at @ThingsExpo, Dr. Alex Gouaillard, CEO and Founder of CoSMo Software, presented ...
I think DevOps is now a rambunctious teenager - it's starting to get a mind of its own, wanting to get its own things but it still needs some adult supervision," explained Thomas Hooker, VP of marketing at CollabNet, in this SYS-CON.tv interview at DevOps Summit at 20th Cloud Expo, held June 6-8, 2017, at the Javits Center in New York City, NY.
CloudEXPO New York 2018, colocated with DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City and will bring together Cloud Computing, FinTech and Blockchain, Digital Transformation, Big Data, Internet of Things, DevOps, AI, Machine Learning and WebRTC to one location.
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of bus...
Everything run by electricity will eventually be connected to the Internet. Get ahead of the Internet of Things revolution. In his session at @ThingsExpo, Akvelon expert and IoT industry leader Sergey Grebnov provided an educational dive into the world of managing your home, workplace and all the devices they contain with the power of machine-based AI and intelligent Bot services for a completely streamlined experience.
DXWorldEXPO | CloudEXPO are the world's most influential, independent events where Cloud Computing was coined and where technology buyers and vendors meet to experience and discuss the big picture of Digital Transformation and all of the strategies, tactics, and tools they need to realize their goals. Sponsors of DXWorldEXPO | CloudEXPO benefit from unmatched branding, profile building and lead generation opportunities.
In his keynote at 19th Cloud Expo, Sheng Liang, co-founder and CEO of Rancher Labs, discussed the technological advances and new business opportunities created by the rapid adoption of containers. With the success of Amazon Web Services (AWS) and various open source technologies used to build private clouds, cloud computing has become an essential component of IT strategy. However, users continue to face challenges in implementing clouds, as older technologies evolve and newer ones like Docker c...
"Evatronix provides design services to companies that need to integrate the IoT technology in their products but they don't necessarily have the expertise, knowledge and design team to do so," explained Adam Morawiec, VP of Business Development at Evatronix, in this SYS-CON.tv interview at @ThingsExpo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
WebRTC is great technology to build your own communication tools. It will be even more exciting experience it with advanced devices, such as a 360 Camera, 360 microphone, and a depth sensor camera. In his session at @ThingsExpo, Masashi Ganeko, a manager at INFOCOM Corporation, introduced two experimental projects from his team and what they learned from them. "Shotoku Tamago" uses the robot audition software HARK to track speakers in 360 video of a remote party. "Virtual Teleport" uses a multip...