Search notes:

Power Query M formula language

The Power Query M formula language is a functional language that allows to express the process of mashing-up data.
Usually, the Power Query M formula language is referred to simply M.
The language resembles F#.
The language is case sensitive.

let expression

let
    variableName    = expression,
    anotherVariable = anotherExpression
in
    yetAnotherVariable
I never understand why the in part is called as it is: it actually specifies the values that are going out.
It seems as though this syntax is borrowed from Haskell (which might have borrowed it from yet another programming language).

Variable names

It's possible to define variable names that contain spaces. They are used with the special #"…" syntax:
let
   #"foo bar baz"  = …
    … 

Literals

Some literals include

Keywords

The keywords include and, as, each, else, error, false, if, in, is, let, meta, not, null, or, otherwise, section, shared, then, true, try, type, #binary, #date, #datetime, #datetimezone, #duration, #infinity, #nan, #sections, #shared, #table and #time.

Immutable values

Values are immutable. The following is not possible, it produces a cyclic reference was encountered during evaluation:
let
   a = 4,
   a = a + 1,
   b = a * 2
in
   a

Comments

M has C / C++ like comments:
/*
   This is
   a multi line comment.
*/

// This comment extends to the end of the line.

Escape character

Within a string, # is the escape character. Some examples are:
#(cr) / #(lf) / #(cr,lf) carriage return / line feed / carriage return, line feed.
#(tab) tabulator
#(263a) arbitrary Unicode character (for example: "Hi #(263a)" = Hi ☺)
In order to get the escape character itself, it needs to escaped like so: #(#).

Two types of values

M deals with two types of values:
Primitive types also include the abstract types: function, table, any and none.
Strings must be enclosed in double quotes, single quotes do not denote a string.

Structured data values (Lists, records and tables)

Lists

A list is denoted by curly braces. The following snippet is a list of the first fibonacci numbers:
{ 1, 1, 2, 3, 5, 8, 13 }

Records

The following is a record, whose two fields are fld_1 and fld_2.
[ 
   fld_1 = 42,
   fld_2 = "hello world"
]

Tables

A table organizes values in columns and rows.
Note that the first list defines the table's column names:
#table(
  {  "col_1","col_2"},
  {
    {    42 ,    11 },
    {    17 ,    32 }
  }
)
See also the #table keyword.

Looking up values in lists or tables

Positional index operator

Individual items in a list can be looked up by the positional index operator ({n}), which accesses the (zero-based) item n
[
   fibonacci = {1, 1, 2, 3, 5, 8, 13, 21, 34},
   five      =  fibonacci{4} // fifth item is 5
]

Accessing values in list of records

[  
  results =   
  {   
      [   
          year       = 2019,   
          val_one    =   42,
          val_two    =   99
      ],  
      [   
          year       = 2020
          val_one    =   47
          val_two    =  103
      ]   
  },  
  sum_val_one = results{0}[val_one] + 
                results{1}[val_one] // equals 42 + 47
]  

Operators

Operators are +, -, *
The meaning of operators depends on the type of its operands.

Concatenation

& concatenates items, such as
  • strings ("hello" & " " & "world")
  • lists ({"foo"} & {"bar", "baz"})
  • records ([ num = 42 ] & [txt = "Hello world"])

Functions

A function takes a set of (input) values and produces a single result value.
The following function multiplies two values:
(a, b) => a * b

each

The each keyword is used to create simple functions with one (the _) parameter.
each [XYZ] is equal to _[XYZ] which is equal to (_) => _[XYZ]

Evaluation model

M's evaluation model resembles that of Excel where values in cells are calculated in order of their dependencies to other cells.

(Standard) library

Like other programming languages, M comes with a standard library, sometimes referred to just as library.

Interesting functions

File.Contents(path as text, optional options as nullable record) as binary
File.Contents return a file's content as binary. This is needed, for example, to open an Excel Workbook and transform it to a table with Excel.Workbook:
Excel.Workbook(workbook as binary, optional useHeaders as nullable logical, optional delayTypes as nullable logical) as table
Table.TransformColumnTypes(table as table, typeTransformations as list, optional culture as nullable text) as table
Table.SelectRows seems to kind of the equivalent of the SQL where clause.
Table.SelectRows(table as table, condition as function) as table
The following line seems to correspond to select * from allRows where colXYZ = 'value ABC':
selectedRows = Table.SelectRows(allRows, each ([colXYZ] = "value ABC")),
Compare with Table.RemoveRows
Table.SelectColumns() / Table.RemoveColumns()
Table.SelectColumns(table as table, columns as any, optional missingField as nullable number) as table
Table.RemoveColumns(table as table, columns as any, optional missingField as nullable number) as table
Table.AddColumn(table as table, newColumnName as text, columnGenerator as function, optional columnType as nullable type) as table
Add a column named added that contains the sum of val one and val two:
sumAdded = Table.AddColumn(inputTable, "added", each [val one] + [val two], type number),
Concatenate two text columns:
concatAdded = Table.AddColumn(inputTable, "concatenated", each Text.Combine({[txt one], " ", [txt two]}), type text),
Table.ReorderColumns(table as table, columnOrder as list, optional missingField as nullable number) as table
Create a table
#table (
  columns as any, // Typically a list of column names, but can also be a table type, a number of columns or null.
  rows    as any  // A list where each element is itself a list that corresonds to a row
)

map equivalent

List.Transform is formula-M's equivalent of the map function that is commonly found in functional programming languages.
List.Transform(list as list, transform as function) as list
The following snippet doubles every element in the list
List.Transform (
{ 11, 3, 29 },
   each 2 * _
)
List.Transform can be used to create lists (rows) from a list. The produced rows can then be returned as a table
let
   numbers = {5, 1, 6, 4, 2, 3},
   rows    =  List.Transform(
                   numbers,
                   each { _ , _*3 , _*_ }
              ),
   table   = #table (
                {"num", "times 3", "squared"},
                  rows
              )
in
   table

Eval

A string can be evaluated (eval'ed) with Expression.Evaluate.
Expression.Evaluate(document as text, optional environment as nullable record) as any 
The following evaluates to 42:
Expression.Evaluate( " 7*6 " )
Passing «symbols» (?) to Expression.Evaluate:
Expression.Evaluate(
"
  f ( {11, 22, val_3 })
",
  [ f     = List.Sum ,
    val_3 = 9
  ]
)

Excel.CurrentWorkbook

Excel.CurrentWorkbook() returns a table that lists the tables of the current Excel workbook. The returned table has the following two columns:
  • Content (which itself is a table)
  • Name (whose data type is string?)
Excel.CurrentWorkbook() as table
Excel.CurrentWorkbook() can be used to refer to named ranges to get values for processing:
var = Excel.CurrentWorkbook(){[Name="rangeName"]}[Content]{0}[Column1],

Record.FieldValues

Record.FieldValues gets values (not the names) from a record.
Record.FieldValues(record as record) as list
The following evaluates to {42, "Hello World", "foo, bar, baz"}.
Record.FieldValues([
   num = 42,
   txt = "Hello World",
   val = "foo, bar, baz"
])
Record.FieldValues might be used to in Table.FromList, but I am not sure what its purpose exactly is:
Table.FromList (
  { 
    [ txt  = "one"  , num = 1 , meta = "foo" ],
    [ WHAT = "two"  , THE = 2 , HECK = "bar" ], // Note the differing record labels!
    [ txt  = "three", num = 3 , meta = "baz" ],
    [ txt  = "four" , num = 4 , meta = "last"]
  },
   Record.FieldValues,    // Function
  {"txt", "num", "meta"}
)

Date and time related

DateTime.LocalNow() evaluates to the current date and time on the system.
DateTime.LocalNow() as datetime
When interested in the data part of a date time, DateTime.Date() is the thing to be used
DateTime.Date(dateTime as any) as nullable date 

Getting data from the internet

Web.Contents(url as text, optional options as nullable record) as binary

Table.FromList

Table.FromList(
      list as list,
      optional splitter    as nullable function,
      optional columns     as any,
      optional default     as any,
      optional extraValues as nullable number
)
as table 
By default, Table.FromList assums the given list a list of text that contains commas where the text needs to be split. Thus, it is almost predestined to read CSV files. The following snippet produes a 4 rows, 3 columns table:
Table.FromList ( { "one,1,foo", "two,2,bar", "three,3,baz", "four,4,last" })
But compare also with Csv.Document().

Table.FromRecords

Table.FromRecords({
     [a = 1, b = 2],
     [a = 3, b = 4]
 }),

Table.PromoteHeaders

Table.PromoteHeaders(table as table, optional options as nullable record) as table
Table.PromoteHeaders is used if a table's first row contains column names:
Table.PromoteHeaders(table as table, optional options as nullable record) as table

Table.Group

Table.Group is used to aggregate records (think the SQL group by clause).
Table.Group(
            table             as          table,       // Data to be aggregated
            key               as          any,         // Non aggregated columns
            aggregatedColumns as          list,        //
   optional groupKind         as nullable number,
   optional comparer          as nullable function
)
as table 

Change values in a given column

Table.ReplaceValue(table as table, oldValue as any, newValue as any, replacer as function, columnsToSearch as list) as table
Some candidates for the replacer function include
  • Replacer.ReplaceValue
  • Replacer.ReplaceText
Table.ReplaceValue can for example be used to replace null values with zeroes, in the following example in the column named foo:
no_nulls = Table.ReplaceValue(chosen_columns, null, 0, Replacer.ReplaceValue, {"foo"})

Select data with SQL with Value.Native

The function Value.Native allows to select data from a database with SQL.
Value.NativeQuery(
             target     as any ,
             query      as text,
    optional parameters as any ,
    optional options    as nullable record
) as any
Because native queries may alter data in the database and are therefore potentially unsafe so that their execution needs to be approved of.
In Excel, under Data -> Get & Transform Data -> Get Data -> Query Options : Security there is a setting to that allows to approve all native queries.
The following example select from an MS Access database:
let
    accdbBinary = File.Contents("P:\ath\to\db.accdb"),
    accdb       = Access.Database(accdbBinary, [CreateNavigationProperties=true]),
    result      = Value.NativeQuery(accdb, "select * from country")
in
    result

TODO

Create a list of dates

// Found at https://blog.crossjoin.co.uk/2016/06/03/creating-tables-in-power-bipower-query-m-code-using-table/

let
  DateList   = List.Dates(#date(2020,1,1), 500, #duration(1,0,0,0)),
  AddColumns = List.Transform(
     DateList,
     each {_, Date.Month(_), Date.Year(_)}
  ),
  DateTable = #table(
                  type table[Date=date, Month=number, Year=number],
                  AddColumns
  
            )
in
  DateTable
// Found at https://blog.crossjoin.co.uk/2016/06/03/creating-tables-in-power-bipower-query-m-code-using-table/

let
   Source = {
       Number.From(#date(2015,1,1))..Number.From(#date(2020,12,31))
   },
   ConvertToTable = Table.TransformColumnTypes(
         Table.FromList(
             Source,
             Splitter.SplitByNothing(),
           {"DateKey"},
             null,
             ExtraValues.Error
         ),
        { {"DateKey", type date}})
in
   ConvertToTable

Misc

Power BI and Formula-M are business analytic tools and therefore not intended to have the ability to write data back to a data source (although there are hacks that seem to make it possible).
Unfortunately, there seems to be no built-in support for regular expressions.

See also

The Formula property of the Excel object WorkbookQuery.
A Hello World example for Formula M.
The workbookQuery object of the Excel Object Model.

Index