7:30am to 8:30am  SQL Server Analysis Services Cube Creation

If you have been curious about Microsoft’s OLAP engine, and how to get started using it, this is the session for you.  In addition to covering the basics of getting, building and navigating the sample OLAP databases, this session will take attendees through a complete build of an SSAS database in our 60-minute, eye-opening session.

11:30am to 12:30 am Ask the Experts Panel, SSAS/MDX/Data Warehousing

I will be working an Ask The Experts session with Bryan Smith.  Here is your chance to play stump the chumps with your questions on all things Microsoft OLAP and Data Warehousing.

2:45pm to 3:45pm College Football and Microsoft Business Intelligence

In my own personal Power Hour session, I will share my passion and fanaticism for college football and data analysis.  The plan is for a light-hearted overview of some fascinating uses of the Microsoft Business Intelligence platform.

After loading nearly 9 Million flights into PowerPivot from the BTS site I constructed DAX formulas for the average delay for the current context filters, across all carriers.  Because I know that I will use this in the context of an Airport, I have labeled it Airport_Avg_Delay. 

The average delay for a given carrier (and/or other context filters) is a simple AVERAGE().

Dividing the AVERAGE by the Airport_Avg_Delay, formatting the result as a percent yields and conditionally formatting the table yields the PivotTable below.  Additionally, I have filtered for only May, 2010 data and only selected airports, for this example.

As I have pointed out earlier, Conditional Formatting is a nice tool for getting a rapid visualization from a table of values.  It can make identification of the best and worst performers easy in a casual glance.

Outliers Matter

The important thing to remember, with out of the box conditional formatting, is that outliers matter.  As more extreme data points are gathered, the degree to which other observations are shaded as ‘hotter reds’ or ‘colder blues’ changes.  For instance, if I were to eliminate Anchorage (ANC), with the American Airlines 375% performance relative to average OR Philadelphia (PHL) with 428% for Skywest, the entire color format of the chart would change.

Another example of why outliers matter is the table below.  I have filtered down to May 10, 2010, which was a bad day to fly into Philadelphia on Skywest or Chicago O’Hare on Pinnacle.  Their performance relative to the average of over 1490% and 2254% skew the entire table.  There is not another red-shaded value of less than 4 times worse than average.  Even performance at double the average is visually indicated as ‘normal’.  Because the range of normal changes with the introduction of the outlier.

ProClarity veterans, and early adopters of PerformancePoint Monitoring and Analytics may remember the Heat Map.  Essentially using size and color of a graphical image to indicate (or map) relative difference in a set of data values.

Using the earlier Percent of Class example, below are a couple of ways to combine a PowerPivot PivotTable and Conditional Formatting to achieve Heat Map-like visualizations.  Given the PivotTable below we highlight the proportion of each conferences players and apply the Conditional Formatting from the Home ribbon menu.  Applying a conditional format takes our table of numbers from this:

to something with more visual cues as to the differences in each value, like this.

Because of the elegance of the DAX formula used to create the Percent of Whole measure, we can easily add Teams to the formatted PivotTable.

I have omitted all but the Southeastern Conference (SEC) from the view in order to illustrate in the next example.  Because our Team table contains both Conferences and Divisions, adding the Division column to the PivotTable creates a handy breakdown and comparison by Conference, Division and Team without any additional formula creation.

While this is not a true Heat Map, I hope you have seen how existing formatting in Excel, combined with PowerPivot analytics can be used to deliver business insight through powerful data visualizations.

• Which stores comprise the greatest share of sales?
• What products make up the least amount of volume?

This sort of ranking and sorting generated complex, correlated sub-queries in the relational world, but are actually quite elegant when solved using an OLAP implementation.  My most recent PowerPivot project involved just such a requirement in a commodity, agri-business.  In appreciation of my client’s confidence, I have literally changed the names to protect the innocent, in order to illustrate the problem and solution.

I have noticed, via Google Analytics and Woopra, blog readers from outside the United States who may be more excited about the World Cup than the Bowl Championship Series.  Unfortunately, this example will make use of American Football, specifically at the collegiate level.  It is my hope the specifics of the game don’t get in the way.  I will make every attempt to ensure the relevant background information is in this post, but please reach out via comment or Twitter if a question should arise.

The Example Problem

American collegiate football players are generally afforded four years of eligibility to play.  There are exceptions for medical reasons, but the sanctioning body for the college game recognizes football student-athletes in the same class names as their student counterparts: Freshman, Sophomore, Junior and Senior.  Because of the speed, power and maturity of the game, Freshman, as a rule are prone to error, slower and not as strong as upper-classmen.  Another complicating factor in player development at this level is most of the highly sought after players have spent their high school years (the four years of school immediately before college) dominating less-skilled populations of players and not being challenged on a game-by-game basis.  At the collegiate level, the competition is far more even from school to school.

Our ‘customer’ would like a PowerPivot measure that calculates the percentage of the whole that each class represents for a school or conference.

The Solution

As part of my presentation for SQL Saturday #28 in Baton Rouge I have been creating examples of Microsoft Business Intelligence tools using NCAA football data.  To illustrate PowerPivot ratio calculations, I am using Roster data from the 2009-2010 season.  Below is an image of the Roster table, including a calculated column to add a number (01-04) prefix to the class year in order to sort slicer and axis values by their progression and not alphabetically.  In other words, Freshman-Sophomore-Junior-Senior and NOT Freshman-Junior-Senior-Sophomore.

To add another dimension to our player data, I have added a Teams table, with the Conference and Division (where applicable) for each school’s team.

Just the DAX

From our simple, Roster and Team data model, creating the PercentofWhole measure is a matter of calculating the number of players in a given class (FR, SO, JR, SR) and the total number of players for the Conference.

So first, the number of players in the class.  The COUNTX function takes two arguments, a Table and an Expression.  In our case (as illustrated below) we are iterating through the ROSTER table counting the Player_ID’s (a unique number assigned each player by the NCAA), to obtain the number of Players.

Using this measure I could easily create a cross-tabulation of Player Class by Conference as illustrated below.  However, other than telling me there are relatively few players in the Independent schools (Notre Dame, Army and Navy), there is not a lot of insight.

So for our next trick, we need to have a means of using the row grand total in a calculation for each class/conference intersection.

ALL(), the Context Remover

In order to have a calculation that is context sensitive for only certain dimensions, the ALL() function can be very useful.  Essentially eliminating a filter context from a measure, it allows us to leverage a total along a given dimension.

For example, our DAX expression for the total of all players below is a combination of two functions.  First (and innermost) is a Count of all Player_ID’s in the Roster table.  Second, this expression is the object of a CALCULATE function that filters the count by ALL values of Sort Year.  It took me a while to get my head wrapped around the idea of a filter that is really every value in the dimension.  Once you realize this, ALL() can be very powerful.

Putting the Numerator and Denominator Together

Because I have created two separate measures (PlayerCount and AllPlayerCount) putting them together to create the desired ratio is crazy simple.

Placing new measure into the PowerPivot PivotTable yields the end result my client was looking for.  A simple cross tabulation of of class year composition by conference.  As I mentioned before, there are some exceptions to the normal Freshman-Senior classifications, and they are accounted for in the column to the left of the 01-FR (Freshman) values.

Because our ALL() is only dealing with the Class Year context, adding Teams to the PivotTable yields the expected result without altering the measure.

Finally, because I have a Division column in the Team table, I can further classify by Division again without altering the measure.

Conclusion

Ratios and composition are a vital part of both business analytics and a feature for which OLAP technologies represent an elegant solution.  I hope that I have illustrated how the ALL() and CALCULATE() functions can be used to precisely control calculation by dimension. 

Coming soon, how to use this technique as the basis for a Heat Map, on the cheap.

The examples for this post use data from the Department of Transportation on flight arrivals and departures, that will be the topic of a more involved post on getting started with PowerPivot.

As you can see in the screenshot below, the chart in the upper right corner of the dashboard represents the percentage of Huntsville departing flights delayed, by destination airport code.  Useful, because by default the order of the x-axis items is in dimension order which is alphabetical in our case.  That is, ATL (Atlanta Hartsfield-Jackson) is before CLT (Charlotte Douglass).

However, our client may want to see this chart in order of the measure value, like the Delay % by Carrier chart to the left.

The key to this little trick is each of our PivotTable charts has behind it a sheet of data.  In the case of this four-chart layout, the charts are numbered top to bottom, left to right.  So, Delay % by Carrier is Chart 1, Delay by Departure Time Block is Chart 2, and our target chart Delay % by Departure City is Chart 3.  The figure below has an arrow from each of the charts to their corresponding data sheet.

Data Sort and Rank Options

Navigating to the Data sheet behind Chart 3, I can quickly sort the charted values and therefore the chart.  Selecting the Destination Airport column, and right-clicking to get a context menu, I can use the menu selections below.

You will notice the default is data source order, hence the original alphabetic order.  We are going to change this to Descending order by the measure value, PercentDelay.

And BAM!  Chart 3 in the Dashboard sheet is now ordered by the measure.  Still showing Delta’s hub and worldwide headquarters Atlanta is worst (for the parameters of this report) but now Houston’s George Bush and Detroit Wayne County have moved Charlotte to fourth, in order order percentage of flights delayed.

Pulling Rank

So the initial steps cover Sorting the chart data.  Now let’s add ranking to the chart.  In the current form, the chart has 11 airports in the x-axis.  Our client just wants to see the top 10, worst airports by percent of flights delayed.  No problem.

From the same data sheet context menu, instead of Sort, we choose Filter and then Top 10.  The result should be a dialog box similar to the one below.  We can actually use this to filter absolute count (top/bottom n), percent (top/bottom 10%) or use an aggregate to filter members from the chart.  In our example, I am leaving the default of 10.

Pressing OK and revisiting the Dashboard sheet, and I see the Chart now has dropped CVG (Cincinnati), to reflect the top 10 airports by percent delay.

Breakfast Basics, SQL Server Analysis Services Cube Creation

The hub of Microsoft’s Business Intelligence platform is Analysis Services.  PerformancePoint Services, Office, and SQL Server Reporting Services all provide out-of-the-box connectivity and features to leverage SSAS data as a first-class data store.  This session is intended to share with participants the basics of creating an SSAS database.  We will cover the relationship between Business Intelligence Developer Studio and SQL Server Management Studio in the cube development and build process.  We will walk through the creation of data sources, to connect to the source of our cube’s data as well as creating data source views and their relationship with the final cube.  How measures, measure groups and partitions are related.  Finally, we will cover on-line editing of an SSAS database using Visual Studio in order to capture metadata that may not be readily available.

For myself, there is no better way to learn than by doing.  The intent of this session is to prepare attendees to build SSAS databases.

College Football and Microsoft Business Intelligence

This session will be a rapid-paced delivery of how Microsoft BI tools (PowerPivot, SQL Analysis Services and PerformancePoint Services) have been used to create a compelling Business Intelligence demonstration using a very non-traditional subject area.  We will tap into readily available NCAA data, conduct comparisons of teams, conferences and players, using the same tools available to business information workers.  All of the source code for the demos will be available for download after the presentation.

With SQL Saturday 28 scheduled only three weeks before the opening of the 2010 college football season, my intention for this session is to both demonstrate how the tools can be used and act as a catalyst in discovering non-traditional BI subject areas.

1. The 3G service in my home market has better performance than AT&T.  (Anecdotal)
2. The 4G service in many of the markets to which I regularly travel is frighteningly fast.
3. Finally, the Sprint 4G markets just happen to be AT&T markets where iPhone data and voice were in effect, useless.

So on to the goodies.  This one device has managed to replace an iPhone 3G, Nikon Coolpix 5100 and Garmin GPS in my travel gear.

The 8-megapixel digital camera and HD (720p) video combine to replace an aging point and shoot.  The Android interface provides quick sharing (or I can offload using the micro-usb cable).  I have not seen the selection of camera apps a la iPhone, but I am sure they will come.

Both the Sprint Navigation (included in my cheaper-than-AT&T plan) and Google gps functions have worked very well on a couple of trips.  I have yet to be steered into a lake or shop window.  Additionally, as the ‘maps’ are always current, no annoying Garmin map updates (or map update fees).

On the downside, the battery life…

Sorry, my Evo battery died in mid-sentence.  Seriously, using the radios (GPS, 4G, WiFi) prudently can get 4-5 hours of use from full charge to exhaustion.  Given most of my GPS use is in a car, the adapter will help offset battery drain for that radio.  I can’t imagine using the WiFi hotspot feature in an area without at least power for the phone (wall or 12-volt).  Finally, I can always buy another battery and keep a spare, charged cell to swap in if needed.  Try that with your iPhone.  So as far as this traveler is concerned.  I am sticking with Sprint and the Evo (at least for two years).

Hold music and introduction slide rotator

Welcome your audience into your web-based auditorium with music appropriate to the target profile.  That may be Metallica, or it could be elevator music.  This important point is there will be something, anything over the audio channel reaffirm to attendees they are in the right place at the right time.

Additionally, I would keep a second presentation, on a perpetual rotation with three slides containing the following:

1. Presentation Title, Sponsor and Speaker (again welcoming your audience)
2. Instructions to enable audio (if they don’t hear music, they won’t hear you).  If you have good web communications/phone integration, provide an alternative telephone conference number for audio-only.  Put the teleconference instructions on this slide.
3. An In Case of Support slide.  Who will your audience contact, and how if they are having persistent issues.

Have a back-channel

It is vital for the presentation team (Marketing, Sales, Technical) all be able to communicate without distracting from the attendee’s experience.  Instant messaging works great for this, but ensure your keyboard clicks are not heard ‘over the air’.

Have a ‘live network feed’ monitor

Just like a network TV control room, you should have a monitor of what is going out over your webinar ‘air’.  This will allow the speaker to time narration better.  Additionally, if a section of the shared content is blocked out (say as Microsoft LiveMeeting can do with shared/unshared content), the speaker will know.

As it is said in show business, ‘break a leg’ on your next webinar delivery.

Back to the reason.  With my passion for public speaking, I do a lot of CodeCamp and SQL Saturday events.  Combine the inevitable (and always welcome) questions that follow a presentation with the NEXT speaker’s need to get setup for the upcoming session and … power supply chaos can ensue.  Marking my laptop power supply with the yellow tape meant I could confidently unplug and pack away my gear while the next presentation ‘began’.  No need to interrupt the presenter to clarify ownership.  The tape’s high visibility comes in very handy in what are usually dimly lit auditoriums.

So it started with marking my power supply, but I do stay more than a few nights in hotels each year.  So a patch of tape here, a wrap there and a the checkout glance around the hotel room is changed.  No more do I donate my iPhone, Kindle, etc charger to the hotel lost and found.

The adhesive used in gaffer tape leaves no residue behind.  If I decide to list my Left Handed 24-port International USB Charger and Surge Protector on craigslist, I can list it in pristine condition.  Additionally, I can inscribe a name, e-mail address or phone number to ensure my gear finds me.

So where do you get it?  I bought mine at Tape Brothers.  I went with fluorescent, because I wanted to see it easily.  I went with the Pro Gaff 1” x 50 yd roll almost two years ago (for $6.25).  I still have 25 yards by my estimation.

How did I choose the exact fluorescent color?  Green didn’t appear to suit my needs for navigating in a dark auditorium, so it was out.  Pink would probably work in the dim lighting, but it is pink.  That left orange and yellow.  No loyal fan of the Alabama Crimson Tide uses orange for anything not mandated by hunting ordinances.  That left yellow.  Roll Tide Yall.

Some images of my gaffer tape usage.