Hi All,
So I'm trying to create run PQ to automate some manual steps that someone else is working on. The data being outputted is only a few thousand rows, however... it's taking about 3 rows a second.
Is there something that can be done to speed it up?? (omitting some info for size-sake, all relevant items still seen)
Original source file is only 512kb .csv file.
prior_data and rep table are about 2,000 row tables in the workbook
date_table is just shy of an 800 row table in the workbook
Here's the code:
let
Source = Csv.Document(File.Contents("\folder_locationsource_file.csv"),[Delimiter=",", Encoding=1252, QuoteStyle=QuoteStyle.None]),
#"Promoted Headers" = Table.PromoteHeaders(Source, [PromoteAllScalars=true]),
#"Changed Type" = Table.TransformColumnTypes(#"Promoted Headers",{{"item1", type text}, {"item2", Int64.Type}, {"item3", Int64.Type}, {"item4", type text}, {"item5", type text}}),
#"Add Prior DWMS" = Table.AddColumn(#"Changed Type", "N_Col1", each prior_data[return this]{List.PositionOf(table2[number],[number])}),
#"Combine DWMS" = Table.AddColumn(#"Add Prior DWMS", "N_Col2", each if [N_Col] = null then [N_Col1] else [N_Col]),
#"Reordered Columns" = Table.ReorderColumns(#"Combine DWMS",{"Col1", "Col2", etc}),
#"Removed Columns" = Table.RemoveColumns(#"Reordered Columns",{"Unwanted Cols"}),
#"Renamed Columns" = Table.RenameColumns(#"Removed Columns",{{"N_Col2", "N_Col}}),
#"Add PLE2" = Table.AddColumn(#"Renamed Columns", "PLE2", each if [PLE] = null then null else if [PLE] < 12 then "0 - <12" else if [PLE] < 24 then "12 - <24" else if [PLE] < 36 then "24 - <36" else if 6 more times else "CHECK"),
#"Reordered Columns1" = Table.ReorderColumns(#"Add PLE2",{"Order new column"}),
#"Renamed Columns1" = Table.RenameColumns(#"Reordered Columns1",{{"Rename columns to match orig table"}),
#"Added Custom" = Table.AddColumn(#"Renamed Columns1", "Sales Rep", each rep[Service Rep]{List.PositionOf(rep[Policy Number],[POLICY_NUMBER])}),
#"Reordered Columns2" = Table.ReorderColumns(#"Added Custom",{"Order new column"}),
#"Replaced Errors" = Table.ReplaceErrorValues(#"Reordered Columns2", {{"Sales Rep", "N/A"}}),
#"Pol_Count for Drv w Trips" = Table.AddColumn(#"Replaced Errors", "Trips", each if ["Trips"]=null then 0 else if [" Trips"]>0 then 1 else 0),
#"Reordered Columns3" = Table.ReorderColumns(#"Pol_Count for Drv w Trips",{"reorder new column"}),
#"Add Week Number" = Table.AddColumn(#"Reordered Columns3", "week_num", each date_table[week_num]{List.PositionOf(date_table[Date],[B_DT])}),
#"Add Bound Year" = Table.AddColumn(#"Add Week Number", "year", each Date.Year([B_DT]))
in
#"Add Bound Year"
Hi Josh,
Welcome to our forum! I'd say the List.PositionOf steps are the bottleneck. You could try wrapping those steps in List.Buffer. This post talks about how to use List.Buffer.
Hope that points you in the right direction.
Mynda