Skip to content

migrated backend in prep to move to 3.12#16

Open
datatalking wants to merge 150 commits into
caiobran:masterfrom
datatalking:master
Open

migrated backend in prep to move to 3.12#16
datatalking wants to merge 150 commits into
caiobran:masterfrom
datatalking:master

Conversation

@datatalking

Copy link
Copy Markdown
Contributor

Created /data, /test, /notebooks and /src folder
Migrated fetch.py and parse.py to src
Moved csv, SQL and data files to /data folder

caiobran added 30 commits March 1, 2019 08:19
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   README.md

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   README.md

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   README.md
	modified:   input/api.json
	modified:   main.py
	modified:   parse.py
	modified:   sql_cmd/select_notupdated1.txt
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   main.py
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	deleted:    test/df_msfinancials.csv
	deleted:    test/df_msratio_cashflow.csv
	deleted:    test/df_msratio_effic.csv
	deleted:    test/df_msratio_finan.csv
	deleted:    test/df_msratio_growth.csv
	deleted:    test/df_msratio_profit.csv
	deleted:    test/msrep_bs_yr_cols.csv
	deleted:    test/msrep_cf_yr_cols.csv
	deleted:    test/msrep_is_yr_cols.csv
	deleted:    test/price_10yr.csv

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	renamed:    sql_cmd/clean.txt -> input/sql_cmd/clean.txt
	renamed:    sql_cmd/delete_fetchedurls.txt -> input/sql_cmd/delete_fetchedurls.txt
	renamed:    sql_cmd/delete_past_past.txt -> input/sql_cmd/delete_past_past.txt
	renamed:    sql_cmd/select_fetched_data.txt -> input/sql_cmd/select_fetched_data.txt
	renamed:    sql_cmd/select_notupdated1.txt -> input/sql_cmd/select_notupdated1-.txt
	new file:   input/sql_cmd/select_notupdated1.txt
	renamed:    sql_cmd/select_notupdated2.txt -> input/sql_cmd/select_notupdated2-.txt
	new file:   input/sql_cmd/select_notupdated2.txt
	renamed:    sql_cmd/select_status_code.txt -> input/sql_cmd/select_status_code.txt
	renamed:    sql_cmd/select_status_code0.txt -> input/sql_cmd/select_status_code0.txt
	renamed:    sql_cmd/select_updated.txt -> input/sql_cmd/select_updated.txt
	renamed:    sql_cmd/update_fetchedurls.txt -> input/sql_cmd/update_fetchedurls.txt
	renamed:    sql_cmd/view_financials.txt -> input/sql_cmd/view_financials.txt
	renamed:    sql_cmd/view_master.txt -> input/sql_cmd/view_master.txt
	modified:   main.py
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	deleted:    input/api0.json
	deleted:    input/ms_financial-report-codes.txt
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   input/sql_cmd/select_notupdated1-.txt
	modified:   input/sql_cmd/select_notupdated1.txt
	modified:   input/sql_cmd/select_notupdated2.txt
	new file:   input/ticker_exch.json
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is ahead of 'origin/master' by 1 commit.
   (use "git push" to publish your local commits)

 Changes to be committed:
	modified:   input/sql_cmd/select_notupdated1.txt
	modified:   input/sql_cmd/select_notupdated2.txt
	modified:   input/tables.json
	modified:   input/ticker_exch.json
	modified:   main.py
	modified:   parse.py
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   input/sql_cmd/select_notupdated1.txt
	modified:   input/ticker_exch.json
	modified:   main.py
	modified:   parse.py
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is ahead of 'origin/master' by 1 commit.
   (use "git push" to publish your local commits)

 Changes to be committed:
	modified:   main.py
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   input/ticker_exch.json
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   input/tables.json
	modified:   input/ticker_exch.json
	modified:   main.py
	modified:   parse.py
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   input/ticker_exch.json
	modified:   parse.py
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	deleted:    _config.yml
	modified:   input/ticker_exch.json
	modified:   main.py
	modified:   parse.py
	modified:   update.py

 Changes not staged for commit:
	modified:   input/ticker_exch.json

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   input/ticker_exch.json
	modified:   parse.py
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	renamed:    input/sql_cmd/select_notupdated1-.txt -> input/sql_cmd/00_archive/select_notupdated1-.txt
	renamed:    input/sql_cmd/select_notupdated2-.txt -> input/sql_cmd/00_archive/select_notupdated2-.txt
	modified:   input/sql_cmd/select_notupdated1.txt
	modified:   input/sql_cmd/select_notupdated2.txt
	modified:   input/ticker_exch.json
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   input/ticker_exch.json
	modified:   parse.py
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   input/ticker_exch.json
	modified:   parse.py
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   input/sql_cmd/select_notupdated1.txt
	modified:   input/sql_cmd/select_notupdated2.txt
	modified:   input/ticker_exch.json
	modified:   parse.py
	modified:   update.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   README.md
	modified:   input/ticker_exch.json
	modified:   main.py
	new file:   tables.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   README.md

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   README.md

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   main.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	renamed:    update.py -> fetch.py
	modified:   input/tables.json
	modified:   input/ticker_exch.json
	modified:   main.py
	modified:   parse.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	renamed:    tables.py -> dataframes.py

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   README.md
	new file:   data_overview.ipynb
	modified:   input/ticker_exch.json

 Untracked files:
	.gitignore
 Your branch is up to date with 'origin/master'.

 Changes to be committed:
	modified:   README.md
	modified:   data_overview.ipynb
	modified:   dataframes.py
	modified:   input/tables.json
	modified:   input/ticker_exch.json
	modified:   parse.py

 Untracked files:
	.gitignore
caiobran and others added 22 commits January 12, 2022 12:42
Depreciated the Morningstar API access

Signed-off-by: Andrew Schell <paintstone@gmail.com>
PR submission to update the repo
PR submission to update the repo, moved fetch.py into root/src/fetch.py
… computing

- Created comprehensive test framework with all test types (database, GPU, multi-machine, automation, etc.)
- Added PostgreSQL and SQLite DandE.db database test suites
- Added GPU acceleration tests (CUDA, PyTorch, LSTM)
- Added multi-machine distributed computing tests
- Created improved SLURM-compatible distributed computing system
- Added Data_Bench integration layer for distributed financial data processing
- Added smoke tests, automation tests, and Airflow placeholder tests
- Created comprehensive test runner script
- Added SLURM integration documentation
- All tests follow DDD pyramid structure (unit, integration, e2e)
- Integrated with nano versioning system and DandE.db tracking
- Created DDD_MERGE_PLAN.md for merging Project_Template and mstables_002
- Created PDS_V2_INTEGRATION_PLAN.md for PDS v2.0.0 critical elements
- Created NEXT_STEPS.md for roadmap and current status
- Updated CHANGELOG.md with DDD merge planning entries
- Documented surgical debugging framework requirements
- Documented anti-hallucination protocol requirements
- Planned DDD restructure (domain/application/infrastructure layers)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants