SQLAuthority News – Links to Book On Line – Data Mining Algorithms (Analysis Services – Data Mining)

I have quite often received request for the Data Mining Algorithms details. Book Online has wonderful resources for the same. I suggest to read them here.

Data Mining Algorithms (Analysis Services – Data Mining)

The data mining algorithm is the mechanism that creates a data mining model. To create a model, an algorithm first analyzes a set of data and looks for specific patterns and trends. The algorithm uses the results of this analysis to define the parameters of the mining model. These parameters are then applied across the entire data set to extract actionable patterns and detailed statistics.

Read more here.

Reference: Pinal Dave (http://blog.SQLAuthority.com)

SQLAuthority News – Webcasts – Resources for IT Managers and their Teams

Pinal Dave and Jacob Sebastian are both SQL Server MVP are doing webcasts for IT Managers and their Teams.

Join us for a 4 series webcast as follows:

Reference: Pinal Dave (http://blog.SQLAuthority.com)

SQLAuthority News – Migrating DTS Packages to Integration Services

Migrating DTS Packages to Integration Services
Writer: Brian Knight
Published: July 2008

SQL Server Integration Services (SSIS) brings a revolutionary concept of enterprise-class ETL to the masses. The engine is robust enough to handle hundreds of millions of rows with ease, but is simple enough to let both developers and DBAs engineer an ETL process. In this whitepaper, you will see the benefits of migrating your SQL Server 2000 Data Transformation Services (DTS) packages to Integration Services by using two proven methods. You will also see how you can run and manage your current DTS packages inside of the SQL Server 2005 and 2008 management tools.

Ready white paper Migrating DTS Packages to Integration Services

Abstract &  courtesy : Microsoft

Reference: Pinal Dave (http://blog.SQLAuthority.com)

SQL SERVER – Future of Business Intelligence

Business Intelligence (BI) is slated to play bigger roles in all kinds of businesses in the coming years. This is not surprising as data analysis and smarter decision making has made the use of BI inevitable in all sizes of businesses across all sectors, including Real estate, IT, mobile devices, governmental agencies, scientific and engineering communities and R&D labs, banking and insurance, to name a few. BI can effectively deal with industry-specific constraints, operations and objectives thereby helping organizations to better understand their customers, optimize their operations, minimize risk, manage revenue, and ultimately improve their results. Moreover, the changing economic environment, which is marked by shrinking budgets, is making way for advancement of successful BI initiatives.

Business Intelligence Forecasting – Predicting Your Way To Success

Today, technology has created one solid platform for the world market, eradicating geographical boundaries. The escalating number of consumers with wide-ranging demands and expectations is making it extremely difficult to conduct business. So meeting customer’s demands has become a top-most priority for all organizations to gain competitive advantage. However, with the implementation of Business Intelligence solutions, companies can stay ahead in the race and keep pace with the market trends and more importantly, meet customers’ expectations. BI Forecasting will help them prepare business strategies while keeping in mind the future events by analyzing the available past data.

Predictive Analytics is the branch of data mining that is used to analyze current and historical data to make predictions about future events. It can help companies optimize their existing processes, well understand customer behavior, spot unexpected opportunities, and anticipate problems before they occur. More and more organizations are realizing the benefits of using data to align their current actions with their future objectives. By incorporating predictive analytics into their daily business operations, they can have better control over the decisions they make every day, which in turn will help them maximize their ROI.

Business Intelligence – Paving Way For Information Democracy

The number of BI users has been constantly growing past the traditional IT “power user”, from dozens, to hundreds and in some cases now thousands of users. Soon this number will be in the millions. Earlier, BI was restricted to only statisticians and corporate analysts and only these selected few could access BI. But today, BI is spreading to nearly every strata of organization, as companies attempt to provide critical data to business users who need it to perform their jobs.

Increasingly, companies are realizing the importance of users having access to timely and relevant insight, which will help in managing performance and aligning them with the mission of the organization. BI is bringing information democracy by providing everyone the insights they require and delivering information to the right people at the right time across the enterprise. More and more organizations are expected to empower their employees with BI for productivity and operational gains.

Software as a Service Business Intelligence –Broadening Usage of BI

A rising number of BI vendors are providing BI software as a service (SaaS) or on-demand business intelligence service. SaaS has been increasingly gaining popularity among small and mid-sized companies. It offers companies a practical option to deploy applications that can provide significant business value. With BI SaaS, companies are not required to make major upfront investments in a hardware server and licenses for the BI software, operating system, web servers and the like. In this case, the software provider hosts them over the Internet for a fee, which can be monthly, quarterly or yearly basis.

The high implementation cost of an end-to-end BI solution was an important factor which discouraged small and mid-sized businesses to adopt BI. With on-demand BI, all sizes of companies can avail the benefits of BI and enhance their business growth. Apart from being cost-effective, it provides several other benefits such as shorter implementation cycles and no maintenance of IT support staff.

Real-time Business Intelligence – Instant Information For Success

Over the last few years BI has been gradually growing in importance and in future, organizations will depend only on real-time information related to their business for smarter decision making. Real-time business intelligence can be defined as the process of delivering information on business operations devoid of any latency. Real-time BI disseminates information about a business in a range from milliseconds to a few seconds after the business event. While traditional business intelligence gives users only historical information, real time business intelligence provides a comparison of present business events with historical events, which helps in identifying a range of issues thereby allowing them to resolve it on time. Moreover, the primary aim of real-time BI is to enable corrective actions to be initiated and business rules to be attuned to optimize business processes.

With consumers demanding faster and improved services from businesses it has become imperative for companies to pay even better attention to consumer satisfaction. They now demand near real-time analysis for intelligent decisions for business success. The rising demand for near-real-time data analysis is making way for a new framework for business intelligence. Business intelligence 2.0 is the recently-coined term to describe the acquisition, provision and analysis of real-time data, which was earlier lacking in BI. BI 2.0 is a part of the constantly developing business intelligence industry and indicates the next step for BI.

BI has truly empowered every businessperson who can now easily use BI to make better decisions and that too on their own, without relying on IT or power analysts to prepare and interpret results for them. In a couple of years BI applications will become as commonplace as spreadsheet applications within all organizations that are midsize or larger. Organizations making effective use of BI technologies will rise and stay far ahead from their competitors. It is expected that BI will soon replace the gut-feel management with real data-based decision-making. Over the coming years, business intelligence will undergo transformation that will have a broad and lasting impact. It will revolutionize the way that we think about business and the way business decisions are made. It’s only when thoughtful analysis supersedes gut feeling and conventional perception, we will enter the next level of business intelligence that will empower businesses with the capacity to reason, prepare, forecast, resolve issues and innovate.

Reference: Pinal Dave (http://blog.sqlauthority.com),DNS

SQL SERVER – Business Intelligence – Aligning Business Metrics

Today, executive management and managers need the latest information to drive intelligent decisions for business success. More informed decisions mean more revenue, less risk, decreased cost, and improved operational control for business agility and competitiveness. Besides, in today’s fast paced, technology-driven business world, organizations are continually struggling to deal with growing data volumes and complexity to use their own data efficiently. Constrained with competitive environments and data complexity are COO, IT Managers and Business Consultants who are asking for less information more easily for smarter, faster decision-making. They want information that is highly visual, up-to-date, personalized and secure. Also, they want information delivered in line with where and how they work.

By leveraging on the power of Business Intelligence (BI) organizations can understand and analyze large volumes of rapidly changing data for effective decision-making. BI helps decision makers to harness the advantage of change to create competitive advantages, achieve corporate objectives, and make better decisions, faster.

Business Intelligence to put it simply, is all about improving decision making within an organization. By presenting the latest information to the right people at the right time the quality of decisions as well as their timeliness can be improved. These days, organizations are embedding BI into business process to create a better workflow, apart from gaining other benefits of BI. In fact, BI can make all areas of business strategic and make them value-rich.

Business Intelligence

Proper BI deployment – Aligning all business metrics

Business Intelligence adds value to all departments or business units within an organization. In general, revenue-generating areas of business such as sales and product manufacturing are often considered critical to a company’s success. While other areas like HR and operations are a requisite to do business but are not considered crucial for revenue generation and for improving business performance. So naturally, when it comes to adding BI value to a business, only revenue-generating departments and areas strike the mind first. However, this is not the case and with appropriate metrics and data this thought process needs to be changed.

Traditionally, Business Intelligence focused primarily on siloed information. Functional areas like Finance, Sales, Marketing and HR created metrics. To provide value to an organization from top to bottom, metrics must align from top to bottom across business units and do away with silos. Undoubtedly, there is still value in maintaining HR, Finance and Sales-specific metrics; however, if the metrics of all areas of an organization are aligned to corporate metrics and objectives, it can prove to be a potent tool to facilitate both the top and bottom lines of an organization. From line leaders to middle managers to executives, they all must have metrics and dashboards that are congruent and support one another.

Nowadays, businesses are putting in a lot of effort to come up with smart, intelligent decisions that will help them run a strategic operation and gain competitive advantage. One of the key aspects of proper Business Intelligence deployment is the alignment of metrics from top to bottom across functional areas, which can go a long way in ensuring business success.

Scenario: Business Intelligence in real world

Let us take a real world scenario and try to understand how proper alignment of business metrics makes all areas of a business more strategic and value-rich.

In our example, let’s take a car manufacturing company Speed Motor Manufacturing. The top priority corporate objective of the company is to increase its profit margin by 5%. To attain this target, it is important to consider each area of business to help impact the bottom line, including the HR department. In general, HR department is considered important by all organizations; however, it is not considered strategic or a contributor to the bottom line.

Now, let’s assume that the operation’s business unit of Speed Motor Manufacturing has ascertained that the best way they can contribute to the overall margin objective is to make 20% more use of a new type of machine which can speed up assembling car components, which will in turn improve the speed of manufacturing process and more cars will be produced per day. However, this complex machine cannot be operated by all individuals. It requires specialized training and more experienced users. Now, in order to meet the objective, the new car manufacturing machine requires an increased usage by 20% for the year.

Here, HR department can play a major role. The HR department ties its goals and metrics to help operations meet their goals, thus directly contributing to the corporate objective of increasing the profit margin by 5%. To meet the objective, HR department must retain the skills required for operating these complex machines, train individuals to operate the new machines and even hire new skills that can operate the new machines. HR’s metrics map specifically to the operations team’s metrics, which map directly to the overall corporate metrics.

In our example, we saw how metrics are aligned top to bottom and across the organization.

Having said that, in order to successfully measure these metrics and create dashboards to check advancement and achievement, the data and the data structure must be available and appropriate. If the data structure is present and the objectives are properly aligned, all aspects of an organization can be strategic and contribute to enhance the business performance for assured success. The HR business unit measures the training it conducted, which increased the number of qualified operators of the new machineries. The operations leader measures the number of hours the new machine was operated compared to the previous year. The finance team measures the profitability of the operations in the current year compared to last year and quantifies operations increase in margin. Thus, we see how all the departments work together towards a common objective, set up metrics to monitor and measure their success and link them accordingly.

In a nutshell, proper deployment of Business Intelligence makes way for perfect objective alignment and improved vision throughout the organization for business success and competitive advantage.

Reference : Pinal Dave (http://blog.SQLAuthority.com), DNS

SQL Server – White Paper – An Introduction to Fast Track Data Warehouse Architectures by Erik Veerman

An Introduction to Fast Track Data Warehouse Architectures
SQL Server Technical Article
Writer: Erik Veerman, Solid Quality Mentors
Technical Reviewer: Mark Theissen, Scotty Moran, Val Fontama
Published: February 2009

The performance and stability of any application solution—whether line of business, transactional, or business intelligence (BI)—hinges on the integration between solution design and hardware platform. Choosing the appropriate solution architecture—especially for BI solutions—requires balancing the application’s intended purpose and expected use with the hardware platform’s components. Poor planning, bad design, and misconfigured or improperly sized hardware often lead to ongoing, unnecessary spending and, even worse, unsuccessful projects.

The ultimate goal of the Fast Track reference configurations is to take the guesswork out of hardware and architectural decisions for the database layer of Microsoft SQL Server-based BI solutions. To help you get started with the Fast Track reference architectures, understand their approach and value, and use them to implement the best possible solution, this paper includes:

  • An overview of the new Fast Track reference architectures
  • A review of BI fundamentals and applicable hardware considerations
  • The tested Fast Track reference architecture components and options
  • Prescriptive guidance for designing and optimizing a solution
  • Resources available to help you choose or create a new hardware configuration

The Fast Track Data Warehouse reference configurations focus on the central database component of a BI solution, commonly called the data warehouse or data mart. These Fast Track reference configurations target solutions on a single server estimated at up to 32TB of data.

Read An Introduction to Fast Track Data Warehouse Architectures

Abstract courtesy : Microsoft

Reference: Pinal Dave (http://blog.SQLAuthority.com)

SQL SERVER – 2008 Star Join Query Optimization

Business Intelligence (BI) plays a significant role in businesses nowadays. Moreover, the databases that deal with the queries related to BI are presently facing an increase in workload. At present, when queries are sent to very large databases, millions of rows are returned. Also the users have to go through extended query response times when joining multiple tables are involved with such queries. ‘Star Join Query Optimization’ is a new feature of SQL Server 2008 Enterprise Edition. This mechanism uses bitmap filtering for improving the performance of some types of queries by the effective retrieval of rows from fact tables.

Improved Query Response Times
In general, data warehouses employ dimensionally modeled star or snowflake schemas. These schemas have one or more than one fact tables that contain transactional data and many dimension tables, which holds information such as product data, customer information, and times and dates – all these define the fact table data. Usually, foreign keys are employed for maintaining relationships between the rows in fact tables and also between the rows in the dimension tables. Databases that contain star schemas are recognized by SQL Server 2008 Enterprise. It uses the new Star Join Query logic for processing queries against such star schemas more efficiently. Typically, on an average, data warehouse queries run faster to approximately 20 percent.

Automatically Implemented
Star Join Query Optimization is automatically implemented by the SQL Server. It does not require a special database or application configuration. The query processor will usually optimize queries with medium selectivity (this refers to the queries that retrieve approximately 10% to 75% of rows from a fact table). Such queries are usually handled using hash joins to join the dimension and fact tables by employing the foreign keys to identify the matching rows. A hash table is built for each dimension table referenced in the query in the case of hash joins; the optimization process uses these hash tables for deriving bitmap filters. The key values from each dimension table are identified by bitmap filters; these key values qualify for inclusion in the query. When the fact table is scanned, the bitmap filters are applied to it. These bitmap filters eliminate those rows of the fact table which are not qualified for inclusion in the result set. The most selective bitmap filter is applied first as it is found to eliminate the highest number of rows. Since the eliminated rows do not need further processing, the subsequent filters need not be applied to them – this way the process becomes more efficient.

Query Selectivity
The performance is enhanced in the case of medium selectivity queries while using bitmap filtering because the rows are filtered before any joins are implemented. Hence, there is a decrease in the number of rows that are processed by each join. Bitmap filtering is not applied when queries are highly selective (i.e., those queries that return less than 10% of the rows in a fact table). In such case, a nested loop join is found to be generally more efficient. Similarly, when the queries are not very selective at all (queries which return more than 75% of the rows in a fact table), bitmap filtering is not applied as there are very few rows to be filtered, and hence, there is no requirement of enhancement in performance in this case.

Integer Data Types
Star join optimization is found to give the highest efficiency when the data type of the columns used in the joins is integer. This feature enables the bitmap filter to be applied as part of the initial table or index scan rather than being used at a later stage in the query plan. Most of the queries are benefited from star join optimization since foreign key relationships are commonly implemented using integer-based alternate key columns.

Reference : Pinal Dave (http://blog.SQLAuthority.com)