#!/usr/bin/env python
# coding: utf-8

# # Creating Subgroup Ranks in Power Query Using Table.AddRankColumn Function
# > Table.AddRankColumn is the latest addition to the M language. This post shows how you can use it to create subgroup rank column 
# 
# - toc: true 
# - badges: true
# - comments: true
# - categories: [powerbi, M]
# - hide: false

# ## Table.AddRankColumn

# Table.AddRankColumn is the latest addition to the M language which lets you create ranks in Power Query directly. No more buffer-sort-add index column trick which was computationally expensive and could be less accurate. With this function you can specify one or more columns to rank by and also the order (ascending/descending). This is not a replacement for the `RANKX` function in DAX. If your rank order is not going to change or need it for validation purposes, this is a great function.  
# 
# Read the [official documentation](https://docs.microsoft.com/en-us/powerquery-m/table-addrankcolumn) for all the function parameters. But Reza Rad has the most [comprehensive overview](https://radacad.com/adding-pre-calculating-rank-in-power-bi-using-power-query) of the function with examples so definitely check it out.

# >note: You need to upgrade to June 2022 version of Power BI to use it in Desktop. 

# ## Subgroup Rank

# In the example below, we have three products, their colors and the Units. `Rank1` column shows the rank by number of units for all the products in the ascending order. What we are looking for is the subgroup rank, i.e rank within each product type as shown in `Rank2` column.  

# In[23]:


import pandas as pd
df = (pd.read_clipboard()
      .assign(Units = lambda s:s['Units'].astype('int'))
      .assign(Rank1 = lambda s:s['Units'].rank().astype('int'))
      .assign(Rank2 = lambda s:s.groupby('Product')['Units'].transform('rank').astype('int'))
      .sort_values(['Product','Rank2'])
           
     )
df


# Here is my walkthrough of how you can create subgroup ranks in Power Query.
# 
# >youtube: https://youtu.be/ysDBHMbtXsk

# #### Important Notes:
# 
# 1. As I mention in the video, this function is **not query foldable** so be sure to plan your steps to utilize folding first. If you don't know what is query folding, [watch my in-depth session](https://player.vimeo.com/video/714177719?h=466f2d1c1b) on query folding. 
# 
# 2. When you create the rank column, it will always sort the table by the newly created rank column. This may not always be desired. For example if your table is sorted by date or a key column when imported, that sort order will be disrupted when you create the rank column. I am not sure if there is a way to prevent that from happening. This also has an unintended effect that vertipaq compression may not be optimal. As you know, cardinality of the column matters in [vertipaq compression](https://www.red-gate.com/simple-talk/databases/sql-server/bi-sql-server/vertipaq-optimization-and-its-impact-on-columnstore-compression/) but distribution of values matters as well. If the sort order is disrupted it may increase the size of the dataset. Be sure to check before and after. 
# 
# 3. I haven't tested this on a large dataset but be sure to check the refresh time and dataset size after creating the rank column. 
# 
# 4. Look at the `RankKind` [parameter](https://docs.microsoft.com/en-us/powerquery-m/rankkind-type) to specify how the ranking should be done.

# In[ ]: