Data Engineering Technical Blog

Summary Visual Studio Online is cloud version of TFS which we can connect using Visual Studio to retrieve projects collections and work items. The VSO does not give access to any backend database like CRM tools and exposes its data through REST API and ODATA. Requirement : Pull project and work item collection from VSO and store the same in SQL Azure, which will be used for analysis. Solution : Create SSIS package to connect to VSO using OData connection and pull VOS data like project , work items etc and connect to SQL Azure and store the desired data. Prerequisite Install Odata for SSIS from (  https://www.microsoft.com/en-us/download/details.aspx?id=42280  ) Create alternate account password in VSO, the same will be used in SSIS Odata connection manager. Steps to create SSIS package Create connection to Azure and VSO Open SSDT and create new SQL Server Integration Service project, add a new package and go to the connection manager and right click, yo...

Step 1: Configuring SQL Reporting Services – Web Service URL Simply go to Reporting Services Configuration Manager and choose Web Service URL and populate the following needed information. The fields are named properly so I guess there is no need for further explanation. What this does is that it configures the IIS for you depending on what Virtual Directory names you had declared. Step 2: Configuring SQL Reporting Services – Create a Report Database Same here, fields need no further explanation except for one which is Native Mode and SharePoint Integrated mode which I will explain below. Choose create a database or if you already have one choose an existing one. For this example, we will create a new one: Connect to the database where you want your Report Data to be stored: Give it a Name and a Report Server Mode. With SharePoint Integrated Mode the report RDLs are stored on SharePoint and not in the Report Database. For this instance, we will use the SharePoint ...

CHANGE THE CONFIGURATION ENTRY IN REPORT SERVER FOLDER ·         Navigate to the report server folder which “C:\Program Files\Microsoft SQL Server \ MSRS_vv.MSSQLSERVER\Reporting Services\ReportServer” ·         Open the rsreportserver.config file and add the below entry below <Configuration> ·         < HTMLViewerStyleSheet > Pink </ HTMLViewerStyleSheet > remember that Pink is will be your CSS file name which we will create in next step. ·         Save and close the file. ADD Pink.css file in the Style folder ·         Open the Styles folder in \ReportServer and create a file named Pink.css ·         Copy the content of HTMLViewer.css to file name Pink.css. ·         Now you can modify the content of Pink.css according to ...

When we think about dimensional modelling then two major and very important modeling schema comes into picture called the start schema and the snowflake schema. So lets go into details on start and snowflake schema. Star schema What is star schema? The star schema architecture is the simplest data warehouse schema. It is called a star schema because the diagram resembles a star, with points radiating from a center. The center of the star consists of fact table and the points of the star are the dimension tables. Usually the fact tables in a star schema are in third normal form(3NF) whereas dimensional tables are de-normalized. Despite the fact that the star schema is the simplest architecture, it is most commonly used nowadays and is recommended by Oracle . Snowflake schema What is snowflake schema? The snowflake schema architecture is a more complex variation of the star schema used in a data warehouse, because the tables which describe the dimensions ...

TYPES OF DIMENSIONS Conformed Dimension: Conformed dimensions mean the exact same thing with every possible fact table to which they are joined. Eg: The date dimension table connected to the sales facts is identical to the date dimension connected to the inventory facts. Junk Dimension: A junk dimension is a collection of random transactional codes flags and/or text attributes that are unrelated to any particular dimension. The junk dimension is simply a structure that provides a convenient place to store the junk attributes. Eg: Assume that we have a gender dimension and marital status dimension. In the fact table we need to maintain two keys referring to these dimensions. Instead of that create a junk dimension which has all the combinations of gender and marital status (cross join gender and marital status table and create a junk table). Now we can maintain only one key in the fact table. Degenerated Dimension: A degenerate dimension is a dimension whic...

Step by step to add custom code in SSRS Introduction SSRS custom code extends re-usability of certain logic in multiple places inside a report. It also helps get rid of writing lengthy calculations in many places with a function. SSRS 2005/2008R2 allows us to write custom codes in VB. If somebody is familiar with C# can choose another option called - Custom Assembly. We will create a sample report with a simple Custom code which will accept two numbers and return the greater of two. So for the sample report I have a given table  Test table with three columns A,B,C Now we will try creating a report from the source data so first we will add a new report in BIDS SSRS project. create proper data source and data set referring to the above table. Now first we will go to Report Property and will click on the Code portion and write the below code. After writing the code we will drag a table and will specify all the columns and will add one extra column that will...

Merge join without SORT Transformation Merge join requires the IsSorted property of the source to be set as true and the data should be ordered on the Join Key. So when we add a SORT transformation it sets the IsSorted property of the source data to true and allows the user to define a column on which we want to sort the data ( the column should be same as the join key). Now to avoid the using SORT transformation we need to set the metadata of the source properly for successful processing of the data else we get error as IsSorted property is not set to true. We will try to join two tables Department and Employee on DeptID column without using SORT transformation in our SSIS package. Department Table details Employee Table details Steps in SSIS package Create a new package and drag a dataflow task. Now right click on Data-flow and click on edit, the data-flow container opens. First task is to create a connection to the database.    Now w...