Next Generation
Microsoft Business Analytics
SelfSelf-Service BI • Corporate BI • Big Data •
Machine Learning
Peter Myers
Bitwise Solutions Pty Ltd
v5 20Mar2015
Presenter Introduction
Peter Myers
Independent BI Expert, Bitwise Solutions
BBus, SQL Server MCSE, MCT, SQL Server MVP (since 2007)
Experienced in designing, developing and maintaining Microsoft database
and application solutions, since 1997
Focuses on education and mentoring
Based in Melbourne, Australia
[email protected]
http://www.linkedin.com/in/peterjsmyers
Seminar Aim
• Provide IT decision makers, IT professionals, business analysts
and data scientists with the information they need to
effectively explore, analyze, visualize and discover insights
from data – structured or unstructured
• Introduce the next generation of Microsoft business analytics:
– Self-Service BI with Microsoft Power BI
– Corporate BI with Microsoft SQL Server
– Cloud-Based Big Data with Azure HDInsight
– Cloud-Based Predictive Analytics with Azure Machine Learning
Seminar Outline
•
•
•
•
•
•
•
Introduction
Introducing Microsoft Business Analytics
Self-Service BI, with Power BI
Corporate BI, with Microsoft SQL Server
Big Data, with Azure HDInsight
Machine Learning, with Azure Machine Learning
Conclusion, Q&A
Logistics
• Please silence mobile phones
• Seminar hours
– Meal breaks
• Restrooms
• Feedback
Presentation Download
• This presentation can be downloaded in PDF from:
– http://www.bitwisesolutions.com.au/downloads/201502/
NextGenerationMicrosoftBusinessAnalytics.pdf
Module Outline
• Introducing Microsoft Business Analytics
• Introducing Microsoft SQL Server
• Defining Roles
Introducing Microsoft Business Analytics
• Data provides organizations with the insight to:
– Analyze its past
– Optimize its present, and
– Make strategic decisions for the future
• The key question is:
In a world of increasing volumes and complexities of data, and
information on demand, how can we provide insight to enable action?
Introducing Microsoft Business Analytics
Strategy and Vision
• To improve organizations by providing
business insights to all employees,
leading to better, faster, more
relevant decisions
– Microsoft has a demonstrated long-term
commitment delivering complete and integrated BI
– SQL Server has led innovation in the BI space for
more than 15 years
– There is widespread delivery of BI through Office
– Solutions can span from on-premises to the cloud
– The platforms are enterprise-grade and affordable
Introducing Microsoft Business Analytics
Microsoft Products and Services
• In line with Microsoft’s vision, Microsoft continues to deliver
products and services to enable powerful data solutions
• These products and services make it easier and faster to gather
and prepare data, and deliver it to the people who need it
Self-Service Business Intelligence, with Microsoft Power BI
Corporate Business Intelligence, with Microsoft SQL Server
Big Data, with Azure HDInsight
Machine Learning, with Azure Machine Learning
Microsoft Power BI
• Power BI is a cloud-based business analytics service (software-as-aservice) for non-technical business users to visualize and analyze data
• Power BI securely connects to a broad set of data sources residing both
on-premises and in the cloud
Azure HDInsight
• Microsoft’s Hadoop-based service that enables Big Data
solutions in the cloud, available as a Microsoft Azure service
• Empowers organizations with new insights on previously
untouched unstructured data, while
connecting to the most widely used
BI tools on the planet
Azure Machine Learning
• Armed with nothing but a browser, professionals can log on to
Azure and start developing prediction models from anywhere
– and deploy new analytic models quickly
• Azure Machine Learning also retains a practically unlimited
number of files on Azure Storage and connects seamlessly with
other Azure data-related services, including:
– HDInsight
– SQL Database, and
– Virtual Machines
Introducing Microsoft SQL Server
• SQL Server is the foundation of Microsoft’s comprehensive
data platform
• It delivers breakthrough performance for mission-critical
applications, using in-memory technologies
• It enables faster insights from any data to any user in familiar
tools like Excel, and a resilient platform for building, deploying,
and managing solutions that
span on-premises and cloud
Introducing Microsoft SQL Server
Introducing Microsoft SQL Server
SQL Server 2014
• SQL Server 2014 enables customers to build mission-critical
applications and Big Data solutions by using highperformance, in-memory technology across different
workloads:
– OLTP
– Data warehousing
– Business Intelligence, and
– Analytics
Introducing Microsoft SQL Server
SQL Server 2014: Breakthrough Data Platform Performance
• Mission critical performance
• Faster insights from any data
• Platform for hybrid cloud
Introducing Microsoft SQL Server
SQL Server 2014: Editions
Introducing Microsoft SQL Server
SQL Server 2014: Complete and Consistent Data Platform
Summary
• Microsoft has a demonstrated long-term commitment
delivering complete and integrated BI
– Self-Service Business Intelligence, with Microsoft Power BI
– Corporate Business Intelligence, with Microsoft SQL Server
– Big Data, with Azure HDInsight
– Machine Learning, with Azure Machine Learning
• SQL Server 2014 enables building mission-critical applications
and Big Data solutions by using high-performance, in-memory
technology across different workloads
Customer Stories
Edgenet
• Edgenet provides optimized product data for suppliers,
retailers, and search engines including Bing and Google
• The company powers its online selling guide with an
In-memory OLTP solution delivered with SQL Server 2014
• By taking advantage of Microsoft in-memory technologies, the
company can provide real-time access to product data, gains
seven times faster throughput, and improves both customer
satisfaction and business insight
https://customers.microsoft.com/Pages/
CustomerStory.aspx?recid=12499
Defining Roles
– IT Professional: A skilled employee who is a member of the IT department and who
creates or maintains corporate BI solutions, or supports managed BI solutions: Database
administrator, developer, or BI developer
– Analyst: A proficient non-IT employee who is empowered to create or maintain SSBI
solutions: Business analyst, or power user
– Data Scientist: A highly educated and proficient employee who can solve complex data
problems by employing deep expertise in some scientific discipline (mathematics, statistics
or computer science)
– User: An employee who has permission and requirements to browse, and interact with, all
types of BI solutions: Information worker, consumer or decision maker
Resources
• Microsoft Power BI
– http://www.powerbi.com
• Microsoft SQL Server
– http://www.microsoft.com/sql
• Azure HDInsight web site
– http://azure.microsoft.com/en-us/services/hdinsight
• Azure Machine Learning web site
– http://azure.microsoft.com/en-us/services/machine-learning
Module Outline
•
•
•
•
•
Defining Self-Service BI
Introducing Power BI
Power BI Authoring
New Power BI Features and Capabilities
New Power BI Pricing
Defining Self-Service BI
• The goal of Self-Service BI (SSBI) is to empower analysts so
that they can design, customize and maintain their own BI
solutions
– Without SSBI, they are otherwise forced to rely entirely upon data
and resources made available by IT
– In reality, though, it is probably happening in an unmanaged way,
whether IT allow it or not
Defining Self-Service BI
(Continued)
• SSBI is not a complete solution nor a replacement for
corporate BI
– IT Pros still need to deliver corporate BI
– So, SSBI is a combination of corporate BI and extensions to empower analysts to
more fully exploit it
– It is not about analysts working in isolation
– Rather, it is about analysts working more closely with IT Pros to share some of
the BI workload
• Used and managed appropriately, it usually proves itself to be
invaluable to the organization
Defining Self-Service BI
Advantages
• The organization becomes more agile, and benefits from the
ability to gather and analyze data more quickly
• IT Pros can remained focused on corporate BI
• Analysts can:
–
–
–
–
–
Access and work with corporate data without reliance on IT
Create ad hoc solutions
Create personal and team-centric solutions
Base decisions on data instead of intuition
Make faster and more accurate decisions
Defining Self-Service BI
Disadvantages
• SSBI can be expensive and time consuming to implement and
maintain
– Analysts will require training, specialized tools and data access
– Metadata dictionaries should be maintained to provide consistent
and reliable access to data
• SSBI queries can impact on corporate system performance and
availability
Defining Self-Service BI
Disadvantages (Continued)
• SSBI solutions can result in duplication of effort, data and logic
– Commonly referred to as “multiple versions of the truth” or
“spreadmart”
• SSBI solutions may not be known to IT, documented, reviewed,
approved, backed up, or adequately secured
– Knowledge of, and expertise to maintain, SSBI solutions could be lost
when an analyst leaves the organization
Defining Self-Service BI
Managed BI
• Managed BI is about mutual benefit for IT Pros and analysts,
and IT governance
• The goal of clear IT governance is to ensure that the
investment in IT generates business value and mitigates risk
Defining Self-Service BI
Managed BI (Continued)
• For SSBI, this means ensuring responsible BI by managing with
oversight to:
–
–
–
–
–
–
Review, approve and audit solutions
Ensure trustworthy data is delivered in a compliant, responsive and secure way
Ensure data, metadata and logic remain available and current, and is backed up
Have visibility into how data is used throughout the organization
Ensure appropriate access permissions are enforced
Incorporate or upgrade SSBI solutions into corporate BI solutions, if appropriate
Defining Self-Service BI
Summary
• IT Pros implement corporate BI, and solutions do not usually
deliver all user information requirements
– IT Pros can remain focused on delivering enterprise requirements,
and can deliver and support a managed BI environment
– A partnership between IT and analysts encourages and supports SSBI
• SSBI is not a replacement for, but an extension of, corporate BI
• A good governance process will increase the adoption of BI
and mitigate risk
Introducing Power BI
• Power BI is a service that empowers anyone to:
– Securely connect and explore data sources
– Create dashboards
– Receive intelligent real-time insights
and proactive alerts
• Solutions can be authored in
Excel, or in the new
standalone designer
(in preview)
Introducing Power BI
• Power BI solutions can be authored with:
– Power Pivot
– Power Query
– Power View
– Power Map
Introducing Power BI
(Continued)
• Once published to a Power BI site, additional sharing and
collaboration features are available:
– Forecasting with Power View
– Natural language queries (in English only) with Power BI Q&A
– Query and data management with the Data Catalog
– Power BI mobile apps
– Cloud-based data refresh, including from on-premises data sources
Introducing Power BI
Power Pivot
• Use Power Pivot to develop an intuitive query-able data
resource that serves business user experiences
• Integrates data from a variety of data sources, including:
– Traditional data source, such as relational databases
– Non-traditional sources, such as data feeds, text files and
spreadsheets
• Delivers accelerated access to extremely
large data volumes
Introducing Power BI
Power Query
• Use Power Query to discover, transform and consume data
• Allows defining queries which run a sequence of steps to
import and reshape data from one or more data sources
– Query steps are defined by using Power Query Formula Language
(also known as “M”)
• Simple query step logic does not require writing formulas
• Advanced query step logic can be written to leverage the full power of the
language
• Includes many data connectors to popular data stores
Introducing Power BI
Power Query: Data Source Types
Introducing Power BI
Power Query: Data Source Types: Continued
Introducing Power BI
Power View
• Use Power View to produce interactive data exploration,
visualization, and presentation experiences
– Highly visual design experience
– Rich meta-driven interactivity
– Presentation-ready at all times
Introducing Power BI
Power View: Example
Introducing Power BI
Power Map
• Use Power Map to interactively visualize spatial data in 3D
• Requires a tabular data model, including a Power Pivot data
model
• Animated tours can be created
and played in the Excel client or
exported to MP4 video
– A soundtrack can be added to the video
• Available only in Excel
Power BI Authoring
• Power BI solutions can be authored in:
– Excel, or
– The new standalone Power BI Designer (desktop)
– The new standalone Power BI Dashboards (browser)
Power BI Authoring
Excel
Access
• Power View
• Power Map
Share
Clean
Visualize
Mash-up
Explore
Power BI Authoring
Next Generation: Power BI Designer
• Standalone offline designer for creating content specifically
for the Power BI Service
• Provides an option for those who do not have access to the
latest version of Excel, giving them the capability to connect
to data and create models and interactive reports for the
service
– Freely downloadable, so you can get started with SSBI at no cost
– Enables connecting to data and publishing data and reports to
Power BI
– Unifies the formerly separate power tools into one enriched user
experience and one Power name
Power BI Authoring
Next Generation: Power BI Designer (Continued)
Power Query
Power Pivot
Power View
New Power BI Features and Capabilities
•
•
•
•
•
•
Power BI dashboards
New out-of-the-box connectors
New data visualizations
Live connectivity to SQL Server Analysis Services
Native mobile apps
Power BI for developers
New Power BI Features and Capabilities
Power BI Dashboards
• Monitor live dashboards
for the data that matters most
• Ask questions of your data
through natural language query
• Drill through to underlying reports
to explore and discover new
insight
• View or author HTML5 responsive
dashboards in a web browser
• Pin new visualizations and KPIs
to monitor performance
• Share dashboards with your team
New Power BI Features and Capabilities
New Out-of-the-box Connectors
• Build new dashboard
with out-of-the-box
connectors for popular
SaaS solutions
New Power BI Features and Capabilities
New Data Visualizations
New Power BI Features and Capabilities
Live Connectivity to SQL Server Analysis Services
Live
Connectivity
• Manage and secure data onpremises with SQL Server
Analysis Services tabular
• Faster time to insight with a
hybrid BI solution
• Optimized query performance for
interactive exploration
• Role-based security is honored
New Power BI Features and Capabilities
Native Mobile Apps
• Native apps for iPad, iPhone,
and Windows devices
• Receive alerts to important
changes in your data
• Share and collaborate with
colleagues and take immediate
action
New Power BI Features and Capabilities
Power BI for Developers
• Create and manipulate Power BI objects via the REST API:
– Dashboards and Reports
– Data source connections and Data Sets
• Package reusable solutions, apps, custom data sources and
add-ins:
– Power BI Packaged Apps
– Power BI Dev Portal
– Power BI REST API
New Power BI Pricing
• Today, you can register to use the preview of Power BI for free
–
Power BI:
• A “freemium” offering
• Limited to 1GB data capacity, and basic features
–
Power BI Pro:
• USD 9.99 per user per month
• 10GB data capacity, and full feature access
Summary
• SSBI is not a replacement for, but
an extension of, corporate BI
• When managed appropriately, SSBI
usually proves invaluable to the
business
• Power BI, with its choice of
authoring tools and cloud services,
enables rapid and agile business
analytics
Immediate
Value
Built-in Best
Practices
Flexible and
Extendable
Elastic
Scale
Customer Stories
• Numerous customer stories are available to learn how Power BI
solutions have been developed to address various industryspecific challenges
Customer Stories
Helse Vest
• Helse Vest is a state-owned regional health authority that manages 10
hospitals in Western Norway
• It can now easily view hospital data, and has cut report time by 93
percent with a cloud BI solution
• By using Power BI for Office 365, it experiences many benefits:
– Employees can easily and quickly view medical data from multiple hospitals
– Analytical reports are created in less than one day (down from up to 14 days)
– The organization can quickly comply with national safety program requirements
https://customers.microsoft.com/Pages/
CustomerStory.aspx?recid=2223
Resources
Power Pivot
• Power Pivot web site
– http://www.microsoft.com/en-us/bi/powerpivot.aspx
• Book: “Microsoft Excel 2013: Building Data Models with PowerPivot”
– Publisher: Microsoft Press
– Authors: Alberto Ferrari and Marco Russo
• DAX Resource Center
– http://social.technet.microsoft.com/wiki/contents/articles/1088.dax-resourcecenter.aspx
Resources
Power Pivot (Continued)
• Whitepaper: DAX in the BI Tabular Model
– Includes a sample Excel workbook
– http://www.microsoft.com/download/en/details.aspx?id=28572
• Book: “DAX Formulas for PowerPivot: The Excel Pro's Guide to Mastering
DAX”
– Publisher: Holy Macro! Books
– Author: Rob Collie
Resources
Power Query
• Microsoft Download Center
– Microsoft Power Query for Excel
– http://www.microsoft.com/en-us/download/details.aspx?id=39379
• TechEd North America 2013
– DBI-B324: “What’s New in Power Query for Excel” by Faisal Mohamood and
Miguel Llopis
– http://channel9.msdn.com/Events/TechEd/NorthAmerica/2014/DBI-B324
• Power Query for Excel Formula Language Specification
– http://go.microsoft.com/fwlink/?LinkID=320633
Resources
Power Map
• Microsoft Download Center
– Power Map Preview for Excel 2013
– http://www.microsoft.com/en-us/download/details.aspx?id=38395
Resources
(Continued)
• Web site: Microsoft Power BI
– http://www.powerbi.com
– Sign up for the free 30 day trial to preview the Power BI service
• MSDN Blog: Power BI
– http://blogs.msdn.com/b/powerbi/
• Power BI Pricing (and feature matrix)
– http://powerbi.com/dashboards/pricing
• Microsoft Customer Power BI Stories
– https://customers.microsoft.com/pages/AdvancedSearch.aspx?SearchText=power bi
Resources
Resources
• Power BI Demo Contest
– http://blogs.msdn.com/b/powerbi/archive/2013/10/16/get-ready-get-set-forthe-power-bi-demo-contest.aspx
– Over 50 imaginative solutions produced with real data by using Power BI
– Finalists: https://www.facebook.com/microsoftbi/app_112813808737465
Get Started Today
Sign up for a free Preview account
Take the Power BI Tour
Read through Getting Started
Knowledgebase and Tutorials
Watch YouTube Videos
Register on the Developer Portal
http://powerbi.com/dashboards
Module Outline
• Implementing Corporate BI
• Introducing the SQL Server BI Services
Implementing Corporate BI
1: Clients
2:
3:
4:
5:
6:
7:
8:
The data
Data
Staging
Manual
sources
warehouse
need
may
use
cleansing
areas
warehouse
various
access
access
can
may
isbe
may
periodically
simplify
data
tools
to
mirrored/replicated
manages
data
be
sources
torequired
query
thedata
populated
data
directly
the
to
for
warehouse
data
cleanse
analyzing
tofrom
warehouse
reduce
dirty
data
population
and
contention
data
sources
reporting
Data Warehouse
Data Sources
Data Marts
Staging Area
Client Access
Manual
Cleansing
Client
Access
Introducing the SQL Server BI Services
End-to-End BI
• Enterprise Information Management
– Integration Services
– Master Data Services
– Data Quality Services
• Analysis Services
• Reporting Services
Enterprise Information Management
“Enterprise Information Management (EIM) is a best practice for creating, managing, sharing,
and leveraging information in an enterprise, holistic manner that’s aligned with strategic,
data-driven business objectives”
– Philip Russom, TDWI
• EIM – more concisely – is about improving the “Four Cs” of data:
–
–
–
–
Completeness
Consistency
Cleanliness
Currency
• SQL Server delivers EIM with three services
Enterprise Information Management
Integration Services
Master Data Services
Data Quality Services
Complete, Consistent, Clean and Current Data
Enterprise Information Management
SQL Server Integration Services
• Primarily designed to implement ETL processes
• Provides a robust, flexible, fast, scalable and extensible
architecture
• Its capabilities are useful in many other scenarios:
–
–
–
–
–
Assessing data quality
Cleansing and standardizing data
Merging data from heterogeneous data stores
Implementing ad hoc data transfers
Automating administrative tasks
Enterprise Information Management
SQL Server Master Data Services
• Master Data Services (MDS) is a product for master data management
delivered in SQL Server
– V1: First released with SQL Server 2008 R2
– V2: The SQL Server 2012 release included many new features and enhancements
– V3: The SQL Server 2014 release included no new capabilities or features
• Delivers credible, consistent data with user-centric data governance
• It addresses several problems:
–
–
–
–
No authoritative source – reports cannot be trusted
No formal dimension maintenance capability
Multiple stakeholders for the same record
Inability to enforce data stewardship processes
Enterprise Information Management
SQL Server Master Data Services: Sample Implementation
Source
System(s)
MDS
Administration
WCF Data
Service
Data Changes
Data
Steward via
MDM or
Excel
SSIS
Outlook
Data
Warehouse
Enterprise Information Management
SQL Server Data Quality Services
• Data Quality Services (DQS) is a knowledge-driven data
cleansing product delivered in SQL Server
• It enables data stewards to cleanse, match, standardize, and
enrich data
• Data cleansing can be performed in a Data Quality Project
– Results are output to a SQL Server table, Excel workbook or CSV file
• Data cleansing can be integrated with:
– Integration Services: DQS Cleansing transform
– Master Data Services: Matching in the Excel MDS Add-in
Analytics with Analysis Services
• Data Modeling (BI Semantic Model)
–
–
–
–
Developed using tabular or multidimensional development approaches
Delivers intuitive browsing and high performance query results
Performs calculations difficult to perform using relational queries
Supports advanced Business Intelligence, including KPIs
• Data Mining component
– Discovers patterns in both relational and OLAP data
– Can enhance the Data Modeling component with discovered results
Analytics with Analysis Services
BI Semantic Model
Power
View
Third-Party
Applications
Reporting
Services
DAX
Excel
Power Pivot
SharePoint
Insights
MDX
Relational
Databases
Files
OData
Feeds
Cloud
Services
Hadoop
Big Data
Deployed
BISMs
Analytics with Analysis Services
BI Semantic Model: One Model For All End User Experiences
Personal BI
Power Pivot for Excel
Team BI
Power Pivot for SharePoint
Corporate BI
Analysis Services
Analytics with Analysis Services
Data Mining Algorithms
Classify
Estimate
• Decision
Trees
• Logistic
Regression
• Naïve
Bayes
• Neural
Networks
• Decision
Trees
• Linear
Regression
• Logistic
Regression
• Neural
Networks
Cluster
• Clustering
Forecast
Associate
• Time Series
• Association
Rules
• Decision
Trees
Reporting with Reporting Services
• Delivers enterprise, web-enabled reporting functionality
• Key attributes:
–
–
–
–
–
–
–
Queries a wide variety of data sources
Publishes reports in various formats
Manages security on content and tasks centrally
Supports pull- or push-driven report delivery
Scales to support thousands of users
Enables extensions to core functionality
Delivers self-service reporting with Report Builder and Power View
Summary
• SQL Server delivers complete end-to-end corporate BI:
Staging
Data Mart
ETL
Master Data Management
Data Quality Assurance
Data Modeling
Data Mining
Managed Reporting
Customer Stories
Yahoo!
• California-based Yahoo! operates one of the most popular websites in
the world
• They implemented a solution that takes data from its vast data stores
within the Apache Hadoop open-source framework and ultimately
processes it in an Analysis Services cube
• 135 gigabytes of data per day is processed by the cube, making it the
largest known SQL Server Analysis Services cube in the world
https://customers.microsoft.com/Pages/
CustomerStory.aspx?recid=14678
Resources
• Microsoft Business Intelligence
– http://www.microsoft.com/en-us/server-cloud/solutions/businessintelligence/default.aspx
• Microsoft Business Intelligence – Analysis
– http://www.microsoft.com/en-us/server-cloud/solutions/businessintelligence/analysis.aspx
• Microsoft Business Intelligence – Predictive Analytics
– http://www.microsoft.com/en-us/server-cloud/solutions/businessintelligence/predictive-analytics.aspx
Resources
(Continued)
• White Paper: “Choosing a Tabular or Multidimensional Modeling
Experience in SQL Server 2012 Analysis Services”
– Free download from Microsoft
– Authors: Liz Vitt, Scott Cameron and Hilary Feier
– http://msdn.microsoft.com/en-us/library/hh994774.aspx
• Book: “Data Mining for SQL Server 2008”
– Authors: ZhaoHui Tang and Jamie MacLennan
– Publisher: Wiley Press
Module Outline
• Introducing Big Data
• Introducing Hadoop
• Introducing Azure HDInsight
Introducing Big Data
Introducing Big Data
“Big Data is a collection of data sets so large and
complex that it becomes awkward to work with
using on-hand database management tools.
Difficulties include capture, storage, search, sharing,
analysis, and visualization.”
– Wikipedia
Introducing Big Data
Big Data solutions deal with complexities of:
VOLUME
(Size)
VARIETY
(Structure)
VELOCITY
(Speed)
Introducing Big Data
Petabytes
Terabytes
Gigabytes
Megabytes
Data Complexity: Variety and Velocity
Introducing Big Data
Responding to New Questions
What’s the social sentiment
of my product?
How do I better predict
future outcomes?
How do I optimize my services
based on patterns of weather,
traffic, etc.?
Introducing Hadoop
• Apache Hadoop is for Big Data
• It is a set of open source projects that transform commodity hardware
into a service that can:
– Store petabytes of data reliably
– Execute huge distributed computations
• Key attributes:
–
–
–
–
–
Open source
Highly scalable
Runs on commodity hardware
Redundant and reliable (no data loss)
Batch processing centric – using a “Map-Reduce” processing paradigm
Introducing Hadoop
Comparison to Traditional RDBMS
TRADITIONAL RDBMS
Data Size
Access
Updates
Structure
Integrity
Scaling
DBA Ratio
HADOOP
Introducing Azure HDInsight
• HDInsight is Microsoft’s Hadoop-based service that enables
Big Data solutions in the cloud
– Available as a Microsoft Azure service
– HDInsight Server is available to install on-premises only for the
purpose of development and testing
• Empowers organizations with new insights on previously
untouched unstructured data, while
connecting to the most widely used
BI tools on the planet
Introducing Azure HDInsight
(Continued)
• Key attributes:
– 100% Apache Hadoop solution, in the cloud
– Built on Hortonworks Data Platform (HDP)
– Map-Reduce logic can be developed with .NET and Java
– Can be automated with PowerShell and Command Line
– Insights can be delivered with Excel Power add-ins
Introducing Hadoop
How it Works
Introducing Hadoop
How it Works
RUNTIME
Server
Server
Server
Server
Introducing the Hadoop Ecosystem
Legend
Red = Core Hadoop
Blue = Data processing
Distributed Storage
(HDFS)
Data Integration
( ODBC / SQOOP/ REST)
Event Pipeline
(Flume)
Distributed Processing
(MapReduce)
Green = Packages
Purple = Microsoft
integration points and
value adds
Orange = Data
Movement
Traditional E-Commerce Data Flow
New E-Commerce Big Data Flow
The Hadoop Data Flow
Data
Hadoop
Analytics
Summary
• Big Data refers to data sets so large and/or complex that they become
awkward to work with in conventional ways
• Hadoop can store petabytes of data reliably and execute huge
distributed computations
– Azure Blog storage provides a persistent and economical data store
– Azure HDInsight provides computational power
• Big Data queries often involve significant latency
– Use traditional BI techniques to cache data in advance of business user queries
• Big Data is just another data source!
– The Power add-ins can query, analyze and visualize Big Data
A Microsoft customer story
describes how Klout produced
a multidimensional BI
Semantic Model (cube) based
on their open-source Hive
data warehouse system
Resources
• Azure HDInsight web site
– http://azure.microsoft.com/en-us/documentation/services/hdinsight/
• Videos: PASS Big Data Virtual Chapter
– https://www.youtube.com/channel/UCkOKmMW_LEsACOqE8C1RWdw
• Hortonworks tutorials
– http://hortonworks.com/tutorials
– Numerous tutorials are available to learn about Big Data by using the
Hortonworks Sandbox
• Klout customer story
– http://www.youtube.com/watch?v=eRXEA9-l2eQ
Module Outline
• Introducing Machine Learning
• Introducing Azure Machine Learning
Introducing Machine Learning
“Machine learning is a subfield of computer science
and statistics that deals with the construction and
study of systems that can learn from data, rather
than follow only explicitly programmed instructions.”
– Wikipedia
Introducing Machine Learning
(Continued)
How can I add
two numbers
together?
f()
num1, num2
Introducing Machine Learning
(Continued)
How can I predict
customer profit?
f()
Age, Marital Status, Gender, Total
Children, Education, Occupation,
Home Owner, Commute Distance
The United States Postal Service
processed over 150 billion pieces of
mail in 2013—far too much for
efficient human sorting
But as recently as 1997, only 10% of
hand-addressed mail was
successfully sorted automatically
The challenge in automation is
enabling computers to interpret
endless variation in handwriting
By providing feedback, the Postal
Service was able to train computers
to accurately read human
handwriting
Today, with the help of machine
learning, over 98% of all mail is
successfully processed by machines
Microsoft
Microsoft
and&Machine
Machine
Learning
Learning
1515
Years
of Realizing
Innovation
years
of realizing
innovation
1999
2000
2004
2008
2010
2012
2014
Computers
work on users
behalf, filtering
junk email
SQL Server
enables data
mining
Microsoft
search engine
built with
machine
learning
Bing Maps
ships with ML
trafficprediction
service
Microsoft
Kinect can
watch users
gestures
Successful,
real-time,
speech-tospeech
translation
Microsoft
launches
Azure Machine
Learning
John Platt,
Distinguished scientist at
Microsoft Research
“
Machine learning is pervasive throughout
Microsoft products.
”
Microsoft
Microsoft
and&Machine
Machine
Learning
Learning
Analytics
Yesterday
15 years
of realizing
High
Competition
innovation
Strategic Change
Expensive
Isolated Data
Lots of Buzz Words
New Markets
Tool Chaos
Complexity
DATA
SCIENTIST
Yesterday‘s
Approach
• Guessing
• Rules of thumb
• Trial and error
Consequences
• Lost opportunities
• Expensive operative
mistakes
Microsoft
Microsoft
and&Machine
Machine
Learning
Learning
Analytics
Today
15 years
of realizing
innovation
Azure Machine Learning
How It Works
Azure Portal
Azure Ops Team
ML Studio
Data Scientists & BI Devs
ML API service
Developers
One Solution for Machine Learning
Web Apps
Mobile Apps
Power BI/Dashboards
ML API service
Developer
Azure Portal &
ML API service
ML Studio
HDInsight
Azure Storage
Azure Ops Team
Data Scientist & BI Dev
Desktop Data
One Solution for Machine Learning
Web Apps
Mobile Apps
ML
service
and the Developer
MLAPI
API
service
Power BI/Dashboards
Business users easily access results:
from anywhere, on any device
Developer
• Tested models available as an url that can be called from any end point
Azure Portal
& ML API
service
Azure
Portal
&
and the Azure Ops Team
ML
APIMLservice
• Create
Studio workspace
• Assign storage account(s)
• Monitor ML consumption
• See alerts when model is ready
Azure
Ops Team
• Deploy models to web service
ML
MLStudio
Studio
HDInsight
and the Data Scientist
Access and prepare data
Create, test and train models
Collaborate
One click to stage for
Data
Scientist
production via the API
service
•
•
•
•
Azure Storage
Desktop Data
Solutions at the Speed of the Market
Manage and Deploy Machine Learning from the Azure Portal
Fully
managed
Easy to use
Tested
solutions
Deploy in
minutes
No software to install,
no hardware to manage,
and one portal to view
and update
Simple drag, drop and
connect interface you
can access and share
from anywhere
Access to sample
experiments, tested
algorithms, support for
custom R and Python,
and over 350 R
packages
Tooled for quick
deployment, hand-off
and updates
Opportunities
Imagine what you could use
Machine Learning for…
Ad
targeting
Churn
analysis
Image
detection &
classification
Equipment
monitoring
Recommendations
Forecasting
Spam
filtering
Fraud
detection
Anomaly
detection
Summary
• Machine Learning is a subfield of computer science and
statistics that deals with the construction and study of systems
that can learn from data
• Azure Machine Learning key attributes:
–
–
–
–
–
Fully managed: No hardware or software to buy
Integrated: Drag, drop and connect
Best in Class Algorithms: Proven solutions from Xbox and Bing
R Built In: Use over 350 R packages, or bring your own R or Python code
Deploy in Minutes: Operationalize with a click
Customer Stories
Pier 1 Imports
• Retailer Pier 1 Imports wanted to better connect with its customers
using insights and data. To do that, the company took to the cloud to
pilot a predictive analytics solution based on Microsoft Azure Machine
Learning and Microsoft Power BI
• As a result of the pilot, Pier 1 Imports may use data insights to predict
which products customers will want in the future, create a dynamic
website using predictive modelling and create more
efficient and effective marketing campaigns
https://customers.microsoft.com/Pages/
CustomerStory.aspx?recid=11257
Customer Stories
Carnegie Mellon University
• Carnegie Mellon University uses Microsoft Azure to reduce
building maintenance and energy costs
• They have used Azure Machine Learning for better fault
detection, diagnosis, and more efficient operations
• With these capabilities, CMU personnel gain advanced
analytics for improved operational insights and decisions,
and gains a way to cut energy use by 20 percent
https://customers.microsoft.com/Pages/
CustomerStory.aspx?recid=8576
Bertrand Lasternas
Carnegie Mellon
We see Azure ushering
in an era of self-service
predictive analytics for
the masses. We can
only imagine the
possibilities
Announcement
• Microsoft have recently
announced an agreement
to acquire Revolution
Analytics, the leading
commercial provider of
software and services for R
• R is the world’s most widely
used language for
statistical computing and
predictive analytics
Resources
• Azure Machine Learning web site
– Sign up for a free trial
– http://azure.microsoft.com/en-us/services/machine-learning
• Machine Learning Blog
– http://blogs.technet.com/b/machinelearning
• Videos: PASS Data Science Virtual Chapter
– https://www.youtube.com/channel/UCqB3xWdwjA9soFV6EOu7qfg
• Revolution Analytics
– http://www.revolutionanalytics.com
Resources
Learning Resources
• Project Botticelli
– Register at http://projectbotticelli.com?pk_campaign=bw2015xyz
– Online videos:
• What is Azure ML
• Introduction to Azure ML
– Training courses
Conclusion
Conclusion
Bitwise Solutions
Training Services
My company offers training in SQL Server BI and Power BI
All training courses in the Microsoft BI Academy program have been
specifically designed to enable students to quickly commence developing
and maintaining state-of-the-art integrated Business Intelligence (BI)
solutions developed by using Microsoft products
SQL Server
EIM (2 days)
SQL Server
SSAS (3 days)
SQL Server
SSRS (3 days)
Office
Power BI (2 days)
Training courses can be delivered remotely, or in-person
http://www.bitwisesolutions.com.au
Presentation Download
• This presentation can be downloaded in PDF from:
– http://www.bitwisesolutions.com.au/downloads/201502/
NextGenerationMicrosoftBusinessAnalytics.pdf
Download

Power BI - Bitwise Solutions Pty Ltd