fix: fct_cost_centers missing table crashes dashboard – graceful degradation + tests#139
fix: fct_cost_centers missing table crashes dashboard – graceful degradation + tests#139
Conversation
…I steps Agent-Logs-Url: https://github.com/godatadriven/github-contributions/sessions/8513b939-dbbd-43b5-a646-96b08e150afd Co-authored-by: jessehouwing <4173387+jessehouwing@users.noreply.github.com>
Agent-Logs-Url: https://github.com/godatadriven/github-contributions/sessions/8513b939-dbbd-43b5-a646-96b08e150afd Co-authored-by: jessehouwing <4173387+jessehouwing@users.noreply.github.com>
|
Copilot can we add a job that rebuilds the database prior to redeploying the web app? And do we have tests that ensure that as the database is built, the correct data is added? And that when the GitHub token or GitHub enterprise variables are unavailable a warning is logged while building the database? |
…s; more tests Agent-Logs-Url: https://github.com/godatadriven/github-contributions/sessions/394d0ea4-1aa6-4b97-9739-552dbae13a40 Co-authored-by: jessehouwing <4173387+jessehouwing@users.noreply.github.com>
|
|
||
| - name: dbt build | ||
| run: dbt build --fail-fast | ||
|
|
…am, reorder download steps Agent-Logs-Url: https://github.com/godatadriven/github-contributions/sessions/394d0ea4-1aa6-4b97-9739-552dbae13a40 Co-authored-by: jessehouwing <4173387+jessehouwing@users.noreply.github.com>
Done in 7c8563b. Here's what was added: 1 — Rebuild database before redeploying ( A new 2 — Tests that verify data is added correctly
3 — Warnings when credentials are unavailable
|
Root cause
When a PR is merged to
main, the deploy workflow fires immediately on thepush: branches: maintrigger and downloads the last successful dbt artifact. If the artifact was built beforefct_cost_centerswas added to the dbt models (the first deploy after the model is introduced), the table won't exist in the served DuckDB file.The old code included
costCentersin theallDataLoadedtruthiness check:When the cost-center query throws
Catalog Error: Table with name fct_cost_centers does not exist!,costCentersstaysundefinedandallDataLoadedis permanentlyfalse— the entire dashboard is stuck in the loading-spinner state.Fix
webapp/src/pages/home/Home.tsxcostCentersinallDataLoadedwith!loadingCostCenters.The page now becomes ready once the cost-center query finishes (whether it succeeded, errored, or returned empty results).
SelectBoxonly whencostCenters !== undefined, so the filter is cleanly hidden when the table is absent.deploy_dashboard_to_github_pages.yml— rebuild database before redeployingA new
rebuild_databasejob runsdbt buildon the self-hosted runner whenever the deploy is triggered by apush/workflow_dispatch, ensuring the database always reflects the latest models before the webapp is deployed. Thebuildjob downloads the freshly-produced artifact from this job. When the deploy is triggered by theworkflow_runfrom "Run DBT (daily)", the rebuild job is skipped and thebuildjob downloads directly from the triggering dbt run's artifact (already fresh). Ahead_branch == default_branchguard on theworkflow_runpath prevents artifacts from non-default branches being deployed. npm dependencies are installed before the DuckDB artifact is downloaded to eliminate the cache-poisoning data-flow path.github_contributions/src/github_contributions/plugin.py— warnings for missing credentialsPlugin.initialize()now emitsWARNING-level log entries (visible even without debug mode) whenGITHUB_TOKENorGH_ENTERPRISE/ENTERPRISEare absent, giving operators an immediate signal in thedbt buildoutput.Tests added to prevent regression
Python –
github_contributions/tests/test_api.pyget_cost_centers: happy path, multiple cost centers, empty logins skipped, empty/missing API responsetest_plugin.pyPlugin._cost_centers_to_df: empty DF when no enterprise, data returned when enterprise is set,_cost_center_userspopulated correctly, empty set when no enterprisetest_plugin.pyTestWarningsOnInitialize: token warning fires, enterprise warning fires, no spurious warnings when both credentials are presenttest_plugin.pyTestLoadMethodSchema:load("cost_centers")returns correct schema without enterprise; returns populated data with enterprise; raisesValueErrorfor unknown resourceWebapp –
webapp/src/hooks/useQuery.test.tspages/home/Home.test.tsxCI wiring
pull_request.yml– newpython_testsjob (pytest,dbt-duckdb==1.7.2) runs on PRs touchinggithub_contributions/**webapp_ci.yml– newRun testsstep (Vitest) added after ESLint on PRs touchingwebapp/**Checklist