{"id":5622,"date":"2022-07-22T10:31:37","date_gmt":"2022-07-22T10:31:37","guid":{"rendered":"https:\/\/www.stl-training.co.uk\/b\/?p=5622"},"modified":"2023-12-30T23:04:09","modified_gmt":"2023-12-30T23:04:09","slug":"how-to-efficiently-cleanse-and-reshape-data-in-power-bi","status":"publish","type":"post","link":"https:\/\/www.stl-training.co.uk\/b\/how-to-efficiently-cleanse-and-reshape-data-in-power-bi\/","title":{"rendered":"How to Efficiently Cleanse and Reshape Data in Power BI"},"content":{"rendered":"<p>You can improve efficiency and accuracy when pulling and transforming data from various sources by using the Query Editor Tool in <a href=\"https:\/\/www.stl-training.co.uk\/syl\/205\/microsoft-business-intelligence-advanced-training-courses.html\">Power BI<\/a>. It\u2019s a marked improvement on using macros in Excel for this function. Read on to see why.<\/p>\n<h2><strong>Why use the Query Editor<\/strong><\/h2>\n<p>The Query Editor is where data sourced from a variety of locations can be transformed before loading data into <a href=\"https:\/\/www.stl-training.co.uk\/sharing\/power-bi-desktop-explained\/85\">Power BI Desktop<\/a> to create visuals. It works by making a connection back to the original data source so that whenever changes are made, they are saves as \u2018Applied Steps.\u2019 Importantly, they are stored in memory. Meaning that if source data is periodically changed or added to, then a simple refresh in Power BI desktop is all that is needed to update any report visuals based on that data. The process of cleansing and reshaping data can include:<\/p>\n<ul>\n<li style=\"list-style-type: none\">\n<ul>\n<li style=\"list-style-type: none\">\n<ul>\n<li>replacing values and errors<\/li>\n<li>removing duplicates<\/li>\n<li>applying Trim to remove unwanted spaces<\/li>\n<li>merging and appending data<\/li>\n<li>adding\/removing columns<\/li>\n<li>unpivoting columns<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<\/li>\n<\/ul>\n<p>One of the most common problems with analysing data is when the column structure is too rigid \u2013 see below:<\/p>\n<p><img decoding=\"async\" class=\"alignnone wp-image-5625 size-full\" title=\"Data\" src=\"https:\/\/www.stl-training.co.uk\/b\/wp-content\/uploads\/2022\/07\/Picture1-Martin-Power-BI.png\" alt=\"\" width=\"752\" height=\"161\" srcset=\"https:\/\/www.stl-training.co.uk\/b\/wp-content\/uploads\/2022\/07\/Picture1-Martin-Power-BI.png 752w, https:\/\/www.stl-training.co.uk\/b\/wp-content\/uploads\/2022\/07\/Picture1-Martin-Power-BI-300x64.png 300w\" sizes=\"(max-width: 752px) 100vw, 752px\" \/><\/p>\n<p>If you needed to produce reports on each city\u2019s first 6 month\u2019s sales, this would be tricky to accomplish given the current structure. However, there is a solution by using \u2018Unpivot Columns\u2019 in Query Editor which effectively restructures the table to produce \u2018Month\u2019 and \u2018Sales\u2019 columns in addition to the \u2018City\u2019 column.<\/p>\n<h2><strong>How to use Query Editor<\/strong><\/h2>\n<ol>\n<li style=\"list-style-type: none\">\n<ol>\n<li style=\"list-style-type: none\">\n<ol>\n<li>In Power BI Desktop, connect to the source data: GET DATA &gt; EXCEL WORKBOOK<\/li>\n<li>Browse to the Excel Workbook and select the data table in the Navigator dialog box<\/li>\n<li>Click on the \u2018Transform Data\u2019 button (bottom right) to launch Query Editor<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<p><img decoding=\"async\" class=\"alignnone wp-image-5626 size-full\" title=\"Power BI\" src=\"https:\/\/www.stl-training.co.uk\/b\/wp-content\/uploads\/2022\/07\/Picture2-Powe-BI-Martin.png\" alt=\"\" width=\"855\" height=\"223\" srcset=\"https:\/\/www.stl-training.co.uk\/b\/wp-content\/uploads\/2022\/07\/Picture2-Powe-BI-Martin.png 855w, https:\/\/www.stl-training.co.uk\/b\/wp-content\/uploads\/2022\/07\/Picture2-Powe-BI-Martin-300x78.png 300w, https:\/\/www.stl-training.co.uk\/b\/wp-content\/uploads\/2022\/07\/Picture2-Powe-BI-Martin-768x200.png 768w\" sizes=\"(max-width: 855px) 100vw, 855px\" \/><\/p>\n<ol>\n<li style=\"list-style-type: none\">\n<ol>\n<li style=\"list-style-type: none\">\n<ol start=\"4\">\n<li>Note the \u2018Applied Step\u2019 section on the right that will expand to include further steps<\/li>\n<li>Now select all the Month Columns, right click on the selection and select \u2018Unpivot Columns\u2019 from the list<\/li>\n<li>Rename the \u2018Attribute\u2019 column \u2018Month (see below)<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<p><img decoding=\"async\" class=\"alignnone wp-image-5628\" title=\"Query Editor\" src=\"https:\/\/www.stl-training.co.uk\/b\/wp-content\/uploads\/2022\/07\/Picture3-Power-BI-Martin.png\" alt=\"\" width=\"375\" height=\"279\" srcset=\"https:\/\/www.stl-training.co.uk\/b\/wp-content\/uploads\/2022\/07\/Picture3-Power-BI-Martin.png 545w, https:\/\/www.stl-training.co.uk\/b\/wp-content\/uploads\/2022\/07\/Picture3-Power-BI-Martin-300x223.png 300w\" sizes=\"(max-width: 375px) 100vw, 375px\" \/>\u00a0 <img decoding=\"async\" class=\"alignnone wp-image-5627\" title=\"efficiency\" src=\"https:\/\/www.stl-training.co.uk\/b\/wp-content\/uploads\/2022\/07\/Picture4-Power-BI-Martin.png\" alt=\"\" width=\"190\" height=\"240\" \/><\/p>\n<p>Notice the new structure and also the additional \u2018Applied Steps\u2019. To complete the process, apply the changes back to Power BI desktop:<\/p>\n<ol>\n<li style=\"list-style-type: none\">\n<ol>\n<li style=\"list-style-type: none\">\n<ol start=\"7\">\n<li style=\"text-align: left\">In the Home tab, select \u2018Close &amp; Apply\u2019<\/li>\n<li style=\"text-align: left\">Create a Column Chart showing Value (sales) against Month<\/li>\n<li style=\"text-align: left\">Check in the Data view to see the updated data and new column structure<\/li>\n<li style=\"text-align: left\">Now add September\u2019s sales in the Excel table. Save and close the file<\/li>\n<li style=\"text-align: left\">Finally in Power BI desktop, select REFRESH in the Home tab and check the chart and data have both updated<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<\/li>\n<\/ol>\n<h2><strong>Conclusion<\/strong><\/h2>\n<p>The Query Editor is a powerful and extremely useful tool allowing you to repeat routine cleansing tasks on your datasets without the need to do it all manually. This tool will therefore help you become more efficient in the way you manage data. In doing so, you will ultimately increase your productivity.<\/p>\n<h2><strong>Further Reading<\/strong><\/h2>\n<p>To find out more about Power BI and when to use Power BI vs. Excel, feel free to have a peruse of the following conversation with one of our trainers: https:\/\/www.stl-training.co.uk\/b\/excel-or-power-bi-which-is-better-for-business-reporting\/<\/p>\n<p>&nbsp;<\/p>\n","protected":false},"excerpt":{"rendered":"<p>You can improve efficiency and accuracy when pulling and transforming data from various sources by using the Query Editor Tool in Power BI. It\u2019s a marked improvement on using macros in Excel for this function. Read on to see why. Why use the Query Editor The Query Editor is where data sourced from a variety [&hellip;]<\/p>\n","protected":false},"author":5,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[23,9,650,18],"tags":[662,649,661],"class_list":["post-5622","post","type-post","status-publish","format-standard","hentry","category-data-visualisation","category-microsoft-office","category-power-bi","category-technology","tag-data-analysis","tag-power-bi","tag-query-editor"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.stl-training.co.uk\/b\/wp-json\/wp\/v2\/posts\/5622","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.stl-training.co.uk\/b\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.stl-training.co.uk\/b\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.stl-training.co.uk\/b\/wp-json\/wp\/v2\/users\/5"}],"replies":[{"embeddable":true,"href":"https:\/\/www.stl-training.co.uk\/b\/wp-json\/wp\/v2\/comments?post=5622"}],"version-history":[{"count":13,"href":"https:\/\/www.stl-training.co.uk\/b\/wp-json\/wp\/v2\/posts\/5622\/revisions"}],"predecessor-version":[{"id":5658,"href":"https:\/\/www.stl-training.co.uk\/b\/wp-json\/wp\/v2\/posts\/5622\/revisions\/5658"}],"wp:attachment":[{"href":"https:\/\/www.stl-training.co.uk\/b\/wp-json\/wp\/v2\/media?parent=5622"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.stl-training.co.uk\/b\/wp-json\/wp\/v2\/categories?post=5622"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.stl-training.co.uk\/b\/wp-json\/wp\/v2\/tags?post=5622"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}