Module:Performances-devel

From Angelina Jordan Wiki
Revision as of 14:01, 28 August 2025 by Most2dot0 (talk | contribs) (Added optional different labelFormular for display of group headers)
The code for this module can be found below its documentation. You can edit the code, edit the documentation, or purge the cache.

NOT FOR PRODUCTION USE!

This is the development version of Module:Performances, which you should use instead of this one.


Work in Progress!!

This module is supposed to provide access to the list of performances Angelina did. At the heart of it is a very generic, configurable table-generator, that could also be used in different contexts.

Usage

Overview

{{#invoke: Performances | createTable       -- createTable is the generic table generator
  |page=Data:Performances.json              -- link to page with JSON data
  |supplements=Data:VideoMetaData.json;url  -- supplemental data sources
  |headers=Song,Date,Type,With,Video        -- List of titles used in the group section headers
  |keys=[[<song>]],{{d|<date>}},<type>,<with>,[<url> <song> - <event>] -- computed data items corresponding to those titles
  |sort=<date>                              -- computed sort keys for sorting outer table
  |sort1=<date>                             -- (optional): outer sortkey, same as inner if omitted
  |filters=date:2018,event:Kongsberg        -- filters to select what items are displayed
  |char_limit=7                             -- The amount of characters of the sort key considered for grouping the sections
  |char_limit1=4                            -- (optional): outer char limit, no outer grouping if ommited 
  |caption=Kongsberg 2018 by Month          -- Title displayed in Header of outer Table
  |group_sort=<date> ! <type> ! <pos>       -- how the group section sub-tables are sorted
  |id=1                                     -- Id needed if multiple tables are generated within a page
}}

Options

page: Provides page that contains JSON formated input data.

supplements (optional): comma seperated list of supplemental data source entries of "supplementalDataPage;keyName:charLimit;targetKeyName", with charLimit and targetKeyName being optional. Data from the supplemental source will be matched based on the keys (if the target's key name is ommitted, the same will be used). The charLimit option shortens the primary key value to this amount of characters. The target data can also be in a form, where the matching key data is used as keys itself instead of value of a "key" field.

headers: Comma seperated list of titles used in the group section headers.

keys: Comma seperated list of computed fields used for the colums in group sections defined by the corrosponding headers. Number of entries should match the number of headers.

Key names representing data are enclosed in "<",">" brackets. Each field can reference multiple keys. Other text will be still interpreted e.g. as templates. For the key's values, substitions can be defined by appending them with colons. A pair of strings will be interpretated as the search and replacement strings for a gsub() substition with lua pattern matching. The special keywords "toUpper", "toLower", and "toTitleCase" will perform the appropriate conversions, "limit:n" will limit the resresult to n characters. Several substitutions can be concatenated.

sort: Computed sort fields for sorting outer table, and for labeling of section names. Same syntax as for keys.

group (optional): Definition of how to label section names differently from 'sort' definition.

char_limit (optional): The amount of characters of the sort key considered for grouping the sections. This will also affect the displayed section name if 'group' is not provided.

filters: Comma seperated list of filters to select what items are displayed. A filter is described by "keyname:values". Alternatives within an filter value item can be described with "/". Negations can be described with "!( . )".

sort1 (optional): Outer sortkey. If ommited, the inner sort key will be used.

group1 (optional): Definition of how to label outer section names differently from 'sort1' definition.

char_limit1 (optional): The amount of characters of the sort key considered for the outer grouping. This will also affect the displayed section name, if 'group1' is ommited.

If ommited, no outer grouping will be performed.

id: A unique Id for each table is needed if multiple tables are generated within a page, to provide a basis for unique section ids.

caption (optional): Title displayed in the header of the outer table

group_sort: Computed field. that determines how the group section sub-tables are sorted. Same syntax as for keys.

Features

The generated table may have one or two levels of grouping, and up to three levels of sorting. It has the follwing structure, with the outer grouping being optional (click on the [n Items] buttons to reveal the inner collapsed tables):

Caption (provided as parameter)
1 (outer grouping level seperator)
A (inner grouping level header) [2 Items]
B (inner grouping level header) [1 Item]
2 (outer grouping level seperator)
C (inner grouping level header) [1 Item]

The inner tables are enclosed in

<div youtube-player-placeholder> ... </div> 

tags, which can optionally be used by Javascript code contained in a Gadget (MediaWiki:Gadget-embeddedYouTubePlayer.js) to attach an embedded YouTube player to each section, that is loaded with a playlist of any YouTube video link contained in that section. Clicking on those links will then also load and start the linked video in that player.

Each grouping separator/header will set an anchor id, so that the table rows can be jumped at with internal references, e.g. C.

There is also code available, that will expand the inner table of a jump target when jumped to.

Use of Cargo queries for input

This was recently added as an alternative source for input. It supports:

  • Multiple Cargo tables with configurable fields
  • Nested child tables (1–3+ levels)
  • List field splitting (# or ,)
  • Optional pretty-printed JSON
  • Flexible configuration in JSON or Lua-like syntax

The getJSON() function generates nested JSON structures from Cargo tables. It is meant to verify that queries produce the right structures before using them in the createTable() function.

All table and field configuration is contained in a single parameter: cargo_query.

{{#invoke:CargoQueryTest|getJSON
 |cargo_query=<JSON or Lua-like table>
 |pretty=true
}}
  • cargo_query – required; configuration of tables, fields, nesting, and root table.
  • pretty – optional; if "true", outputs indented JSON (requires mw.text.JSON_PRETTY).

cargo_query Parameters

Parameter Type Description
tables array of strings List of Cargo table names (CamelCase). Order does not determine root.
fields table Map of tableName → comma-separated list of field names to return. Fields can be lowercase.
where table Map of tableName → Cargo WHERE clause for filtering rows. Empty string for no filter.
nest array of tables Each table describes a parent-child nesting rule: { parent = "ParentTable", child = "ChildTable", parentKey = "ParentField", childKey = "ChildField", as = "ChildLabel" }
listFields array of strings Field names that should be returned as arrays (split on # or ,).
root string The Cargo table to use as the top-level JSON object.

Field Splitting Rules

  • # delimiter → splits Cargo lists into an array, preserves empty entries.
 Example: "Concert##US Concert" → ["Concert","","US Concert"]  
  • ', delimiter → splits only if there are no spaces around the comma.

Example 1 — Two-level nesting (Performances → Videos)

{{#invoke:CargoQueryTest|getJSON
 |cargo_query={
   "tables": ["PerformancesDevel","VideosDevel"],
   "fields": {
     "PerformancesDevel": "song,event,context,date,type,pos,partners,comment,perfID",
     "VideosDevel": "perfID,url,duration,quality"
   },
   "where": {
     "PerformancesDevel": "",
     "VideosDevel": ""
   },
   "nest": [
     { "parent": "PerformancesDevel", "child": "VideosDevel", "parentKey": "perfID", "childKey": "perfID", "as": "videos" }
   ],
   "listFields": ["context","partners"],
   "root": "PerformancesDevel"
 }
 |pretty=true
}}

Lua-like table style (sanitized)

{{#invoke:CargoQueryTest|getJSON
 |cargo_query={
   tables = ['PerformancesDevel','VideosDevel'],
   fields = {
     PerformancesDevel = 'song,event,context,date,type,pos,partners,comment,perfID',
     VideosDevel = 'perfID,url,duration,quality'
   },
   nest = {
     { parent = 'PerformancesDevel', child = 'VideosDevel', parentKey = 'perfID', childKey = 'perfID', as = 'videos' }
   },
   listFields = ['context','partners'],
   root = 'PerformancesDevel'
 }
 |pretty=true
}}

Example 2 — Three-level nesting (Songs → Performances → Videos)

{{#invoke:CargoQueryTest|getJSON
 |cargo_query={
   tables = ['SongsDevel','PerformancesDevel','VideosDevel'],
   fields = {
     SongsDevel = 'songID,title,artist',
     PerformancesDevel = 'song,event,context,date,type,pos,partners,comment,perfID',
     VideosDevel = 'perfID,url,duration,quality'
   },
   nest = {
     { parent = 'SongsDevel', child = 'PerformancesDevel', parentKey = 'songID', childKey = 'song', as = 'performances' },
     { parent = 'PerformancesDevel', child = 'VideosDevel', parentKey = 'perfID', childKey = 'perfID', as = 'videos' }
   },
   listFields = ['context','partners'],
   root = 'SongsDevel'
 }
 |pretty=true
}}

Tips

  1. Always specify root in cargo_query to avoid ambiguity.
  2. Use listFields for any Cargo field that should become a Lua/JSON array.
  3. pretty=true is optional but recommended for debugging and readability.
  4. Lua-table style is allowed inline — single quotes ' are fine; the module sanitizes them into valid JSON.
  5. Multi-level nesting must specify both parent and child explicitly in each rule.
  6. Use the AJW:Cargo_query_test page to develop your query and check if it delivers the expected JSON equivalent structure.

ToDo

  • Make sorting more flexible, e.g. by applying regex substitutions to data values before being sorted.
  • Have an option, to not start collapsed (usefull for small tables)
  • Implement access to sublists below keys.
  • provide default values for empty fields of computed fields, e.g. like <keyName=defaultValue>
  • Provide a means to print additional data behind the sort field that does not effect the id-anchor

Examples

Multiple Filters, combined as AND

{{#invoke: Performances-devel | createTable
  |page=Data:Performances.json
  |headers=Song,Date,Type,With,Video
  |keys=[[<song>]],{{d|<date>}},<type>,<with>,[<url> <song> - <event>]
  |sort=<date>
  |filters=date:2018,event:Kongsberg
  |char_limit=7
  |caption=Kongsberg 2018 by Month
  |group_sort=<date> ! <type> ! <pos>
  |id=1
}}
Kongsberg 2018 by Month
2018-06 [12 Items]
2018-07 [22 Items]


Single Event with Supplemental Video Metadata

{{#invoke: Performances-devel | createTable
  |page=Data:Performances.json
  |supplements=Data:VideoMetaData.json;url:43
  |headers=Song,Date,Type,Video
  |keys=[[<song>]],{{d|<date>}},<type>,[<url> <url-title>; <url-User>]
  |sort=<event>
  |group_sort=<date> ! <type> ! <pos> ! <song>
  |filters=event:Bjerke
  |id=2
}}

Alternatives in Filters, combinded as OR

{{#invoke: Performances-devel | createTable
  |page=Data:Performances.json
  |headers=Song,Date,Type,Comment
  |keys=[[#<song>|<song>]],{{d|<date>}}: <pos>,<type> <duration> [<url> play],<comment>; <with>
  |sort=[[<event>]]
  |caption=Repetitive Live Events
  |group_sort=<date> ! <type> ! <pos> ! <song>
  |filters=event:Allsang på Grensen/TV 2's Artist Gala
  |id=3
}}
Repetitive Live Events
Allsang på Grensen [7 Items]
TV 2's Artist Gala [2 Items]

Demonstrating Different Outer Sorting/Grouping and Negation in Filters

In the following example, the highest level (sort1) is sorted by year, then the middle one (sort) by date, as well as the inner table one, which also features secondary sorting keys (type, pos, song). The seperator "!" that was chosen for these has no special meaning, but it is the lowest character encoding value above a space, which should ensure that the correct sorting is applied.

"duration:!(fragment)" is used for filtering for those videos, that are not fragments.

Also, wie use substitutions to the computed "url" field

{{#invoke: Performances-devel | createTable
  |page=Data:Performances.json
  |headers=Song,Date,Type,Comment,Video
  |keys=[[#<song>|<song>]],{{d|<date>}}; <pos>,<type> <duration>, <comment>; <with>,[<url> <url:.*www.: :.com.*: >]
  |sort=[[<event>]]
  |sort1=<date>
  |char_limit1=4
  |filters=type:live,duration:!(fragment)
  |caption=Live Events by Year, excluding fragments
  |group_sort=<date> ! <type> ! <pos> ! <song>
  |id=4
}}
Jump to… 2013 – 2014 – 2015 – 2016 – 2017 – 2018 – 2019 – 2020 – 2021 – 2022 – 2023 – 2024
Live Events by Year, excluding fragments
2013
VAD Festival 1 Dec 2013 [1 Item]
2014
Allsang på Grensen [1 Item]
Christmas 2014 NRK TV [1 Item]
Concert for UN summit 2014 [1 Item]
Ein Herz für Kinder [1 Item]
FMC Technology (Kongsberg Jazz Festival) 2014 [3 Items]
Good Morning Norway [1 Item]
Kongsberg Jazz Festival 2014 [2 Items]
New Year's celebration 2014/15 [1 Item]
Norway's Got Talent [5 Items]
Nyhetsmorgon [1 Item]
Orkla Growth Awards [1 Item]
Prime Minister [3 Items]
Save the Children Peace Prize Party [2 Items]
Schwedish Heros [2 Items]
Senkveld [1 Item]
Shopping center, Norway 2014 [1 Item]
Sommerkveld Med Redningsskøyta [1 Item]
The View [1 Item]
TV 2's Artist Gala [1 Item]
Unknown concert [3 Items]
2015
Encontro com Fátima Bernardes [1 Item]
Hagekonsert på Aksept [1 Item]
Hemnes Båt & Fjord Festival [10 Items]
Jotun Plant Opening [3 Items]
Lindmo [1 Item]
Mediacorps Campus Opening [1 Item]
Victoria Scene, Oslo [1 Item]
2016
100k Follower Facebook live [1 Item]
Alan Walker Is Heading Home [2 Items]
Allsang på Grensen [1 Item]
Clarion Hotel [1 Item]
God Sommer Norge [1 Item]
Instagram Live Stream [2 Items]
Kongsberg Jazz Festival 2016 [10 Items]
Lille Julaften [1 Item]
Little Big Shots [1 Item]
Little Big Shots UK [1 Item]
Rotary Club Convention 2016, Seoul [3 Items]
Salvation Army's Christmas Pot [2 Items]
The Stream Gir Tilbake [1 Item]
TV 2's Artist Gala [1 Item]
Wish Concert [1 Item]
2017
Allsang på Grensen [2 Items]
Christmas concert tour 2017 [32 Items]
Dojima River Forum [1 Item]
Little Big Stars [1 Item]
Muse in City [8 Items]
New Year's concert 2017 [3 Items]
Nor Shipping Conference, Oslo [2 Items]
NRK Radio Interview 2017 [3 Items]
Summer concert tour 2017 – Larkollen [16 Items]
Summer concert tour 2017 – Prøysenfestivalen [13 Items]
Summer concert tour 2017 – Sandefjord [12 Items]
2018
Christmas concerts 2018 [19 Items]
Concert in LA [2 Items]
It's Magic concert [14 Items]
Jake and Rachel's Wedding [2 Items]
Kongsberg Jazz Festival 2018 [13 Items]
Lyden av Norge, P4 Radio [1 Item]
Quincy Jones 85th birthday concert [2 Items]
Skavlan [1 Item]
TechnipFMC Jazzfest (Kongsberg Jazz Festival 2018) [3 Items]
2019
America's Got Talent, The Champions 2 [3 Items]
ÅrsteinØYA 2019 [1 Item]
Summer concert tour 2019 – Prøysenhuset [14 Items]
2022
20th Unforgettable Gala [2 Items]
Allsang på Grensen [2 Items]
Kongsberg Jazz Festival 2022 [15 Items]
Kurbadhagen, Sandefjord [31 Items]
2023
A tribute to Narges Mohammadi (Noble Peace Price 2023) [2 Items]
Germany's Next Top Model 2023 [1 Item]
Le Bal Paris 2023 [1 Item]
Mathias Sorum's birthday party [2 Items]
PM International's President's Gala, Mannheim Germany [1 Item]
Portsmouth 2023 [4 Items]
The Best FIFA Football Awards™ 2022 [1 Item]
2024
Christmas Special 2024 [19 Items]
Glendale 2024 [21 Items]
IVARS Awards Backstage [1 Item]
Jesse & Friends @ The Hotel Cafe [2 Items]
Las Vegas 2024 [38 Items]
Sofar London, The Hoxton Hotel Shoreditch [1 Item]
St. Pancras station, London [4 Items]
The Other Songs, London Palladium [1 Item]
Wish Bus USA [2 Items]
Jump to… 2013 – 2014 – 2015 – 2016 – 2017 – 2018 – 2019 – 2020 – 2021 – 2022 – 2023 – 2024

Demonstrating Same Type Sorting, with Different Grouping Levels, and Supplemental Data

Here, the different char_limits on the sorting of song lead to two different grouping levels, by 1st character and by unique title.

{{AtoZ}}
{{#invoke: Performances-devel | createTable
  |page=Data:Performances.json
  |supplements=Data:Songs.json;song;title,Data:VideoMetaData.json;url:43  |headers=Event,Date,Type,Video,Pos,With,Comment
  |keys=[[#<event>|<event>]],{{d|<date>}},<type> <duration>,[<url> play <song-type>],<pos>,<with>,<comment>; <url-channelName>
  |sort=[[<song>]]
  |char_limit1=1
  |caption=Live Songs, excluding fragments
  |group_sort=<date> ! <type> ! <pos>
  |filters=duration:!(fragment),type:live
  |id=5
}}
{{AtoZ}}

A · B · C · D · E · F · G · H · I · J · K · L · M · N · O · P · Q · R · S · T · U · V · W · X · Y · Z

Live Songs, excluding fragments
7
7th Heaven [3 Items]
A
All I Ask [2 Items]
All I Want for Christmas Is You [6 Items]
All of Me (jazz standard) [5 Items]
All of Me (John Legend song) [1 Item]
At Last [4 Items]
B
Back to Black [4 Items]
Bad Valentine [5 Items]
Bang Bang (My Baby Shot Me Down) [15 Items]
Bohemian Rhapsody [11 Items]
Born to Die [1 Item]
C
Can't Help Falling in Love [9 Items]
Careless Whisper [1 Item]
Cheek to Cheek [1 Item]
Crazy (Willie Nelson song) [12 Items]
Cry Me a River [5 Items]
D
Diamond [2 Items]
Diamonds Are Forever [5 Items]
Dream a Little Dream of Me [3 Items]
Driving Home for Christmas [1 Item]
E
Ev'ry Time We Say Goodbye [4 Items]
F
Faded [5 Items]
Feeling Good [12 Items]
Fly Me to the Moon [30 Items]
G
Gloomy Sunday [1 Item]
Goodbye Yellow Brick Road [4 Items]
H
Have Yourself a Merry Little Christmas [2 Items]
Here Comes the Sun [1 Item]
Hero [2 Items]
Hurt [1 Item]
I
I'd Rather Go Blind [1 Item]
If I Were a Boy [5 Items]
I Got You (I Feel Good) [1 Item]
I Have Nothing [5 Items]
I Just Want to Make Love to You [4 Items]
I'll Be There [6 Items]
I'm a Fool to Want You [2 Items]
I'm Not the Only One [1 Item]
I Put a Spell on You [9 Items]
I Saw Mommy Kissing Santa Claus [6 Items]
It's a Man's Man's Man's World [3 Items]
It's Magic [2 Items]
It's Now or Never [2 Items]
It's the Most Wonderful Time of the Year [2 Items]
I Will Always Love You [1 Item]
J
Jingle Bell Rock [1 Item]
K
Killing Me Softly with His Song [3 Items]
L
La Bohème [1 Item]
Last Christmas [1 Item]
La Vie en rose [1 Item]
Let It Snow! Let It Snow! Let It Snow! [3 Items]
Let Me Entertain You [1 Item]
Love Don't Let Me Go [6 Items]
Love in the Dark [1 Item]
Love on the Brain [2 Items]
M
Mary's Boy Child [5 Items]
Mercy [5 Items]
Million Miles [5 Items]
Million Years Ago [1 Item]
Moon River [1 Item]
Mount Everest [3 Items]
Mr. Lonely [1 Item]
My Funny Valentine [3 Items]
My Way [2 Items]
N
Now I'm the Fool [4 Items]
O
One Moment in Time [2 Items]
Oslo [3 Items]
Our Day Will Come [6 Items]
Over the Rainbow [1 Item]
P
Princess of Ruins [4 Items]
R
Rehab [1 Item]
Right Here Waiting [1 Item]
S
Santa Claus Is Comin' to Town [1 Item]
Set Fire to the Rain [1 Item]
Shield [4 Items]
Silent Night [6 Items]
Sing Me to Sleep [1 Item]
Someone You Loved [1 Item]
Something's Got a Hold on Me [4 Items]
Speak Softly, Love [2 Items]
Stay [3 Items]
Strangers in the Night [2 Items]
Summertime [10 Items]
Suspicious Minds [6 Items]
T
Tears Dry on Their Own [3 Items]
Thank You [2 Items]
The Christmas Song [5 Items]
The House of the Rising Sun [3 Items]
These Boots Are Made for Walkin' [1 Item]
U
Unchained Melody [6 Items]
Unchain My Heart [1 Item]
V
Valerie [8 Items]
W
We're the Best [1 Item]
What a Diff'rence a Day Made [14 Items]
What Am I to You [1 Item]
What a Wonderful World [6 Items]
What Is Life [5 Items]
When We Were Young [6 Items]
When You're Smiling [3 Items]
When You Wish Upon a Star [4 Items]
Where Do Broken Hearts Go [2 Items]
White Christmas [9 Items]
Y
You Don't Own Me [2 Items]
You Raise Me Up [1 Item]

A · B · C · D · E · F · G · H · I · J · K · L · M · N · O · P · Q · R · S · T · U · V · W · X · Y · Z

Code

[Edit module code]


local p = {}

-- Load required libraries
local mw_text = mw.text
local mw_title = require('mw.title')

-- Function to fetch JSON from a wiki page
function p.fetchJSONFromPage(pageName)
    local title = mw_title.new(pageName)
    if not title then
        return nil, 'Invalid page name'
    end
    local content = title:getContent()
    if not content then
        return nil, 'Page not found or empty'
    end
    return mw_text.jsonDecode(content)
end

-- Function to find the matching data in the secondary table
local function findInSecondary(secondaryData, key, secondaryFieldName)
    if type(secondaryData) == "table" then
        -- Case 1: Secondary data is a dictionary with key-value pairs
        if secondaryData[key] then
            return secondaryData[key]
        end

        -- Case 2: Secondary data is a list where keys are the links
        for _, item in pairs(secondaryData) do
            if item[secondaryFieldName] == key then
                return item
            end
        end
    end

    return nil
end

-- Function to limit the key length for comparison
local function limitKeyLength(key, maxChars)
    if maxChars then
        return string.sub(key, 1, tonumber(maxChars))
    end
    return key
end

-- Function to enrich primary data with secondary data
local function enrichPrimaryWithSecondary(primaryData, secondaryPage, primaryFieldName, maxChars, secondaryFieldName)
    secondaryFieldName = secondaryFieldName or primaryFieldName  -- Default to primaryFieldName if secondaryFieldName is not provided

    -- Load the secondary JSON data using fetchJSONFromPage
    local secondaryData, err = p.fetchJSONFromPage(secondaryPage)
    if not secondaryData then
        return nil, "Failed to load JSON data from the secondary page: " .. secondaryPage .. " (" .. err .. ")"
    end

    -- Iterate through the primary data
    for _, item in ipairs(primaryData) do
        local primaryFieldValue = item[primaryFieldName]
        if primaryFieldValue then
            -- Limit the primary field value's length if maxChars is specified
            local limitedPrimaryFieldValue = limitKeyLength(primaryFieldValue, maxChars)

            -- Find the matching data in the secondary table
            local secondaryMatch = findInSecondary(secondaryData, limitedPrimaryFieldValue, secondaryFieldName)
            if secondaryMatch then
                -- Merge the secondary data into the primary data with new key names
                for k, v in pairs(secondaryMatch) do
                    if k ~= secondaryFieldName then  -- Avoid duplicating the linkage field
                        local newKey = primaryFieldName .. "-" .. k
                        item[newKey] = v
                    end
                end
            end
        end
    end

    return primaryData
end

-- Helper for filter tasks 
local function matchesFilter(value, pattern)
    -- Replace '*' with '.*' for wildcard matching
    local luaPattern = pattern:gsub("%%", "%%%%"):gsub("%*", ".*")

    -- Check for group negation (e.g., "!(pattern1/pattern2)")
    local isGroupNegated = luaPattern:match("^%!%b()$")
    if isGroupNegated then
        luaPattern = luaPattern:sub(3, -2)  -- strip "!()"
    end

    -- Split alternatives by '/'
    local alternatives = {}
    for alt in luaPattern:gmatch("[^/]+") do
        -- Escape Lua magic chars in each alternative
        alt = alt:gsub("([%^%$%(%)%%%.%[%]%*%+%-%?])", "%%%1")
        table.insert(alternatives, alt)
    end

    -- Check matches
    local subMatch = false
    for _, alt in ipairs(alternatives) do
        if tostring(value):match(alt) then
            subMatch = true
            break
        end
    end

    -- Apply negation logic if necessary
    if (not isGroupNegated and subMatch) or (isGroupNegated and not subMatch) then
        return true
    end
    return false
end

-- Functions to compute the value of a field based on a formula with optional gsub or transformations
-- Helper: resolve dotted paths with optional [n] indices
local INVISIBLE_SEPARATOR = " "
-- U+2063 character

-- resolve nested paths with optional [n] indices
local function resolvePath(tbl, path)
    local current = tbl
    for segment in path:gmatch("[^.]+") do
        local key, index = segment:match("^(%w+)%[(%d+)%]$")
        if key then
            if current and current[key] then
                local idx = tonumber(index)
                if current[key][idx] ~= nil then
                    current = current[key][idx]
                else
                    return INVISIBLE_SEPARATOR
                end
            else
                return ""
            end
        else
            current = current and current[segment]
            if current == nil then return "" end
        end
    end
    return current
end

-- computeField supporting nested paths, |filter, :transform/substitution
function p.computeField(item, formula)
    return (formula:gsub("(%b<>)", function(placeholder)
        local content = placeholder:sub(2, -2)

        -- Split by '|' for optional filter
        local fieldPart, filterPart = content:match("([^|]+)|?(.*)")
        filterPart = filterPart ~= "" and filterPart or nil

        -- Split transformations/substitutions by ':'
        local parts = {}
        for part in fieldPart:gmatch("[^:]+") do
            table.insert(parts, part)
        end

        local fieldPath = parts[1]
        local value = resolvePath(item, fieldPath)
        value = tostring(value or "")

        -- Apply filter if present
        if filterPart and not matchesFilter(value, filterPart) then
            return ""
        end

        -- Apply transformations/substitutions (rest of fieldPart after first segment)
        if #parts > 1 then
            for i = 2, #parts, 2 do
                local pattern = parts[i]
                local replacement = parts[i + 1]

                if pattern == "toUpper" then
                    value = value:upper()
                elseif pattern == "toLower" then
                    value = value:lower()
                elseif pattern == "toTitleCase" then
                    value = value:gsub("(%a)(%w*)", function(first, rest)
                        return first:upper() .. rest
                    end)
                elseif replacement then
                    value = value:gsub(pattern, replacement)
                end
            end
        end

        return value
    end))
end
-- Function to split a string by a delimiter
local function splitString(str, delimiter)
    local result = {}
    for match in (str .. delimiter):gmatch("(.-)" .. delimiter) do
        table.insert(result, match)
    end
    return result
end

-- Function to filter data based on provided filters with support for alternatives and negation
function p.filterData(data, filters)
    local filteredData = {}
    for _, item in ipairs(data) do
        local match = true
        for key, pattern in pairs(filters) do
            if not matchesFilter(item[key], pattern) then
                match = false
                break
            end
        end
        if match then table.insert(filteredData, item) end
    end
    return filteredData
end

-- Function to sort data based on a sorting key formula
function p.sortData(data, sortKeyFormula)
    table.sort(data, function(a, b)
        local aKey = p.computeField(a, sortKeyFormula):gsub('[^%w%?%!]', ''):lower():gsub('%?',"%#")
        local bKey = p.computeField(b, sortKeyFormula):gsub('[^%w%?%!]', ''):lower():gsub('%?',"%#")
        return tostring(aKey) < tostring(bKey)
    end)
    return data
end

-- Function to group data using a key formula (for boundaries) and an optional label formula (for display)
function p.groupData(data, keyFormula, labelFormula, charLimit)
    local groupedData = {}
    local currentKey = nil
    for _, item in ipairs(data) do
        local keyValue = p.computeField(item, keyFormula)
        if charLimit and tonumber(charLimit) then
            keyValue = mw.ustring.sub(keyValue:gsub('[%(%)%[%]%{%}]','') , 1, tonumber(charLimit))
        end
        local headerValue = labelFormula and p.computeField(item, labelFormula) or keyValue
        if keyValue ~= currentKey then
            currentKey = keyValue
            groupedData[#groupedData + 1] = {header = headerValue, items = {}}
        end
        table.insert(groupedData[#groupedData].items, item)
    end
    return groupedData
end

-- Function to render table headers
function p.renderHeaders(headers)
    return '! ' .. table.concat(headers, ' !! ') .. '\n'
end

-- Function to render table rows
function p.renderRows(items, computedKeys)
    local rows = ''
    for _, item in ipairs(items) do
        rows = rows .. '|-\n'
        local row = {}
        for _, keyFormula in ipairs(computedKeys) do
            row[#row + 1] = p.computeField(item, keyFormula) or ''
        end
        rows = rows .. '| ' .. table.concat(row, ' || ') .. '\n'
    end
    return rows
end

-- Function to render the complete table
function p.renderTable(data, headers, computedKeys, sortKeyFormula, groupLabelFormula, charLimit, sortKeyFormula1, groupLabelFormula1, charLimit1, groupSortKeyFormula, caption, tableId)
    local outerTable = '{| class="wikitable" style="width: 100%; margin: 0;"\n'
    outerTable = outerTable .. '|-\n! colspan="' .. #headers .. '" style="text-align:left;" | ' .. caption .. '\n'

    local sortedData1 = p.sortData(data, sortKeyFormula1)			 -- outer level
    local groupedData1 = p.groupData(sortedData1, sortKeyFormula1, groupLabelFormula1, charLimit1) -- this should be more flexible with flexibile charcount

    for groupIndex1, group1 in ipairs(groupedData1) do
	    local sortedData = p.sortData(group1.items, sortKeyFormula)  -- inner level
	    local groupedData = p.groupData(sortedData, sortKeyFormula, groupLabelFormula, charLimit)
	    outerTable = outerTable ..'|- \n| style="text-align:center;" | <span id="' .. group1.header:gsub('%[%[([^%]|]+)|([^%]]+)%]%]', '%2'):gsub('%[%[([^%]]+)%]%]', '%1') .. '">' .. group1.header .. '</span>\n'
	    
	    for groupIndex, group in ipairs(groupedData) do
	        if groupSortKeyFormula then
	            p.sortData(group.items, groupSortKeyFormula)
	        end
	        
	        local groupId = tableId .. '-' .. groupIndex1 .. '-' .. groupIndex
	        local itemCount = #group.items
	        local toggleLabel = '[' .. itemCount .. ' Item' .. (itemCount > 1 and "s" or "") .. ']'
	        local toggleSpan = '<span class="mw-customtoggle-' .. groupId .. '" style="cursor:pointer; color:blue;">' .. toggleLabel .. '</span>'
	
	        outerTable = outerTable .. '|-\n! colspan="' .. #headers .. '" style="background-color:#f5f5f5; text-align:left;" | <span id="' .. group.header:gsub('%[%[([^%]|]+)|([^%]]+)%]%]', '%2'):gsub('%[%[([^%]]+)%]%]', '%1') .. '">' .. group.header .. '</span> ' .. toggleSpan .. '\n'
	        outerTable = outerTable .. '<tr class="mw-collapsible mw-collapsed" id="mw-customcollapsible-' .. groupId .. '" style="display: none">\n'
			outerTable = outerTable .. '| <div class="youtube-player-placeholder">\n'  -- this would be the place to possibly reduce the padding for the inner table
	        outerTable = outerTable .. '{| class="wikitable sortable" style="width: 100%; margin: 0;"\n'
	        outerTable = outerTable .. '|-\n' .. p.renderHeaders(headers)
	        outerTable = outerTable .. p.renderRows(group.items, computedKeys)
	        outerTable = outerTable .. '|}\n</div>\n'
	        outerTable = outerTable .. '</tr>\n'
	    end
	end
    outerTable = outerTable .. '|}\n'
    return outerTable
end

-- Main function to create the table
function p.createTable(frame)
    local args = frame:getParent().args
    for k, v in pairs(frame.args) do
        args[k] = v
    end

    local pageName = args['page']
    local supplements = args['supplements']

    if not args['headers'] or not args['keys'] or not args['sort'] then
        return 'Error: headers, keys, and sort arguments are required.'
    end

    local headers = mw_text.split(args['headers'], ',')
    local computedKeys = mw_text.split(args['keys'], ',')
    local sortKeyFormula = args['sort']
    local groupLabelFormula = args['group']      -- inner label (optional)
    local charLimit = args['char_limit']
    local sortKeyFormula1 = args['sort1']
    local groupLabelFormula1 = args['group1']    -- outer label (optional)
    local charLimit1 = args['char_limit1']
    local groupSortKeyFormula = args['group_sort']
    local caption = args['caption'] or ''
    local tableID = args['id']

    local filters = {}
    local filterString = args['filters']
    if filterString then
        for key, pattern in filterString:gmatch('([^:]+):([^,]+),?') do
            filters[key] = pattern
        end
    end

    if not pageName then
        return 'Error: page argument is required to fetch JSON data.'
    end

    local primaryData, err = p.fetchJSONFromPage(pageName)
    if not primaryData then
        return 'Error fetching JSON from page: ' .. err
    end

    -- If supplements parameter is provided, process each supplement
    if supplements then
        local supplementsList = mw_text.split(supplements, ",")
        for _, supplement in ipairs(supplementsList) do
            -- Split the supplement string using semicolons
            local parts = mw_text.split(supplement, ";")
            local secondaryPage = parts[1]

            -- Further split the primary field name to extract maxChars if specified
            local primaryFieldParts = mw_text.split(parts[2], ":")
            local primaryFieldName = primaryFieldParts[1]
            local maxChars = primaryFieldParts[2]  -- This will be nil if not specified

            local secondaryFieldName = parts[3] or nil

            -- Enrich the primary data with the current secondary data
            local enrichedData, err = enrichPrimaryWithSecondary(primaryData, secondaryPage, primaryFieldName, maxChars, secondaryFieldName)
            if not enrichedData then
                return "Error: " .. err
            end
        end
    end

    local data = p.filterData(primaryData, filters)

	if not sortKeyFormula1 then
		sortKeyFormula1 = sortKeyFormula
	end
	if not charLimit1 then
		charLimit1 = 0
	end
	
    return p.renderTable(data, headers, computedKeys, sortKeyFormula, groupLabelFormula, charLimit, sortKeyFormula1, groupLabelFormula1, charLimit1, groupSortKeyFormula, caption, tableID)
end

function p.test(frame)
	local data = {
	    { name = "John", status = "first" },
	    { name = "Jane", status = "second" },
	    { name = "Doe", status = "third" },
	}
	
	local filters = {
	    status = "!(first/second)"
	}
	
	local filteredData = p.filterData(data, filters)

	for _, item in ipairs(filteredData) do
	    mw.log(item.name .. ": " .. item.status)
	end
end

return p