diff --git a/README.md b/README.md index 7a4b26ba..c0457519 100644 --- a/README.md +++ b/README.md @@ -388,10 +388,10 @@ even if it fails. You can find your test results in https://github.com/citusdata/release-test-results under `periodic_job_results` folder. Test results will be pushed to a branch which is in the format ${rg_name}/${month_day_year_uniqueID}. -By default the tests will be run against `release-9.2` and the latest released version. If you want to test on a custom branch you should change the config files of relevant tests with your custom branch name in: +By default the tests will be run against `release-11.1` and the main branch. If you want to test on a custom branch you should change the config files of relevant tests with your custom branch name in: ```text -postgres_citus_versions: [('12.1', 'your-custom-branch-name-in-citus'), ('12.1', 'release-9.1')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'your-custom-branch-name-in-citus')] ``` *Note*: While you can run multiple tests by adding more elements to the array above, the results of the tests after the first might @@ -514,7 +514,7 @@ If you want to run only the tpcc benchmark or the analytical queries, you should You can change the thread count and initial sleep time for analytical queries from `build-and-run.sh` with `CH_THREAD_COUNT` and `RAMPUP_TIME` variables respectively. -If you want to run hammerdb4.0 change `hammerdb_version` to `4.0` in `create-run.sh`. +If you want to run hammerdb4.5 change `hammerdb_version` to `4.5` in `create-run.sh`. By default a random region will be used, if you want you can specify the region with `AZURE_REGION` environment variable prior to running `create-run.sh` such as `export AZURE_REGION=westus2`. @@ -558,7 +558,7 @@ fab add.shards-on-coordinator On the coordinator node: ```bash -# This will run default pgBench tests with PG=12.1 and Citus 9.2 and 8.3 release branches +# This will run default pgBench tests with PG=15.1 and Citus release-11.1 and main branches # and it will log results to pgbench_results_{timemark}.csv file # Yes, that's all :) You can change settings in fabfile/pgbench_confs/pgbench_default.ini fab run.pgbench-tests @@ -573,7 +573,7 @@ fab run.pgbench-tests --config-file=pgbench_default_without_transaction.ini On the coordinator node: ```bash -# This will run scale tests with PG=12.1 and Citus 9.2 and 8.3 release branches +# This will run scale tests with PG=15.1 and Citus release-11.1 and main branches # and it will log results to pgbench_results_{timemark}.csv file # You can change settings in files under the fabfile/pgbench_confs/ directory fab run.pgbench-tests --config-file=scale_test.ini @@ -598,7 +598,7 @@ test_count: specifies total test scenarios which uses any ex [main] -postgres_versions: ['14.5'] +postgres_versions: ['15.1'] extensions: ['citus', 'hll', 'topn', 'tdigest', 'auto_explain'] test_count: 4 ``` @@ -653,7 +653,7 @@ conf_string: ''' On the coordinator node: ```bash -# This will run default extension tests with PG=14.5 +# This will run default extension tests with PG=15.1 # Yes, that's all :) You can change settings in fabfile/extension_confs/extension_default.ini fab run.extension-tests @@ -675,7 +675,7 @@ test_count: specifies total test scenarios which uses any ex [main] -postgres_versions: ['14.5'] +postgres_versions: ['15.1'] extensions: ['citus', 'hll', 'topn', 'tdigest', 'auto_explain'] test_count: 4 ``` @@ -730,7 +730,7 @@ conf_string: ''' On the coordinator node: ```bash -# This will run default extension tests with PG=14.5 +# This will run default extension tests with PG=15.1 # Yes, that's all :) You can change settings in fabfile/extension_confs/extension_default.ini fab run.extension-tests @@ -760,7 +760,7 @@ fab run.pgbench-tests --config-file=pgbench_cloud.ini --connectionURI='postgres: On the coordinator node: ```bash -# This will run TPC-H tests with PG=12.1 and Citus 9.2 and 8.3 release branches +# This will run TPC-H tests with PG=15.1 and Citus release-11.1 and main branches # and it will log results to their own files on the home directory. You can use diff to # compare results. # You can change settings in files under the fabfile/tpch_confs/ directory @@ -812,7 +812,7 @@ cd azure # run.valgrind check-multi-vg runs "make check-multi-vg" # run.valgrind check-multi-1-vg runs "make check-multi-1-vg" # run.valgrind check-columnar-vg runs "make check-columnar-vg" -tmux new -d "fab use.postgres 12.3 use.citus release-9.2 run.valgrind check-multi-vg" +tmux new -d "fab use.postgres 15.1 use.citus release-11.1 run.valgrind check-multi-vg" # simply exit from coordinator after detaching @@ -864,8 +864,8 @@ into the VM-s. On the coordinator node: ```bash -# an example usage: Use PostgreSQL 14.2 and run valgrind test on citus/release-9.2 -fab use.postgres 14.2 use.citus release-9.2 run.valgrind check-multi-vg +# an example usage: Use PostgreSQL 15.1 and run valgrind test on citus/release-11.1 +fab use.postgres 15.1 use.citus release-11.1 run.valgrind check-multi-vg ``` However as valgrind tests take too much time to complete, we recommend you to run valgrind tests in a detached session: @@ -875,7 +875,7 @@ However as valgrind tests take too much time to complete, we recommend you to ru # run.valgrind check-multi-vg runs "make check-multi-vg" # run.valgrind check-multi-1-vg runs "make check-multi-1-vg" # run.valgrind check-columnar-vg runs "make check-columnar-vg" -tmux new -d "fab use.postgres 12.1 use.citus release-9.2 run.valgrind check-multi-vg" +tmux new -d "fab use.postgres 15.1 use.citus release-11.1 run.valgrind check-multi-vg" ``` After the tests are finished (takes up to 9 hours with default coordinator size), re-connect to the coordinator. @@ -898,14 +898,14 @@ install Citus: - `fab --list` will return a list of the tasks you can run. - `fab setup.basic-testing`, will create a vanilla cluster with postgres and citus. Once this has run you can simply run `psql` to connect to it. -- `fab use.citus v7.1.1 setup.basic-testing` will do the same, but use the tag `v7.1.1` when installing Citus. You can give it any git ref, it defaults to `master`. -- `fab use.postgres 10.1 setup.basic-testing` lets you choose your postgres version. -- `fab use.citus release-9.2 setup.citus` will install postgres and the `release-9.2` branch of the citus repo. +- `fab use.citus v11.1.5 setup.basic-testing` will do the same, but use the tag `v11.1.5` when installing Citus. You can give it any git ref, it defaults to `main`. +- `fab use.postgres 15.1 setup.basic-testing` lets you choose your postgres version. +- `fab use.citus release-11.1 setup.citus` will install postgres and the `release-11.1` branch of the citus repo. ## Tasks, and Ordering of Tasks -When you run a command like `fab use.citus v7.1.1 setup.basic-testing` you are running two -different tasks: `use.citus` with a `v7.1.1` argument and `setup.basic-testing`. Those +When you run a command like `fab use.citus v11.1.5 setup.basic-testing` you are running two +different tasks: `use.citus` with a `v11.1.5` argument and `setup.basic-testing`. Those tasks are always executed from left to right, and running them is usually equivalent to running them as separate commands. For example: @@ -922,10 +922,10 @@ have an effect on the current command: ``` # this works: -fab use.citus v7.1.1 setup.basic-testing +fab use.citus v11.1.5 setup.basic-testing # this does not work: -fab use.citus v7.1.1 # tells fabric to install v7.1.1, but only works during this command -fab setup.basic-testing # will install the master branch of citus +fab use.citus v11.1.5 # tells fabric to install v11.1.5, but only works during this command +fab setup.basic-testing # will install the main branch of citus ``` `use` tasks must come before `setup` tasks: @@ -933,7 +933,7 @@ fab setup.basic-testing # will install the master branch of citus ``` # this does not work! # since the `setup` task is run before the `use` task the `use` task will have no effect -fab setup.basic-testing use.citus v.7.1.1 +fab setup.basic-testing use.citus v11.1.5 ``` Finally, there are tasks, such as the ones in the `add` namespace, which asssume a cluster @@ -948,9 +948,9 @@ These tasks configure the tasks you run after them. When run alone they have no Some examples: ``` -fab use.citus v7.1.1 setup.basic-testing -fab use.citus release-9.2 setup.citus -fab use.debug-mode use.postgres 10.1 use.citus v7.1.1 setup.basic-testing +fab use.citus v11.1.5 setup.basic-testing +fab use.citus release-11.1 setup.citus +fab use.debug-mode use.postgres 15.1 use.citus v11.1.5 setup.basic-testing ``` `use.debug-mode` passes the following flags to postges' configure: `--enable-debug --enable-cassert CFLAGS="-ggdb -Og -g3 -fno-omit-frame-pointer"` @@ -970,7 +970,7 @@ For a complete list, run `fab --list`. As described [above](#fab-tasks), you can run these at the same time as you run `setup` tasks: -- `fab use.citus v7.1.1 setup.citus add.shard_rebalancer` does what you'd expect. +- `fab use.citus v11.1.5 setup.citus add.shard_rebalancer` does what you'd expect. ## `pg` Tasks diff --git a/azure/azuredeploy.parameters.json b/azure/azuredeploy.parameters.json index 2a9178f7..cd28dbd8 100644 --- a/azure/azuredeploy.parameters.json +++ b/azure/azuredeploy.parameters.json @@ -3,7 +3,7 @@ "contentVersion": "1.0.0.0", "parameters": { "numberOfWorkers": { - "value": 1 + "value": 2 }, "diskType": { "value": "Premium_LRS" diff --git a/fabfile/extension_confs/extension_default.ini b/fabfile/extension_confs/extension_default.ini index ff88c036..5fded578 100644 --- a/fabfile/extension_confs/extension_default.ini +++ b/fabfile/extension_confs/extension_default.ini @@ -1,5 +1,5 @@ [main] -postgres_versions: ['14.5', '15beta4'] +postgres_versions: ['14.6', '15.1'] extensions: ['citus', 'hll', 'topn', 'tdigest', 'auto_explain', 'bloom', 'adminpack', 'amcheck', 'btree_gin', 'btree_gist', 'citext', 'cube', 'dblink', 'dict_int', 'dict_xsyn', 'earthdistance', 'file_fdw', 'fuzzystrmatch', 'hstore', 'intarray', 'isn', 'lo', 'ltree', 'pageinspect', 'passwordcheck', 'pgcrypto', 'pgstattuple', 'pg_surgery', 'pg_trgm', 'pg_visibility', 'postgres_fdw', 'seg', 'tablefunc', diff --git a/fabfile/hammerdb_confs/citus_config.ini b/fabfile/hammerdb_confs/citus_config.ini index d84ba152..44c9a022 100644 --- a/fabfile/hammerdb_confs/citus_config.ini +++ b/fabfile/hammerdb_confs/citus_config.ini @@ -1,5 +1,5 @@ [DEFAULT] -postgres_citus_version: ('14.5', 'main') +postgres_citus_version: ('15.1', 'main') postgresql_conf: [ # the following two are necessary to prevent getting timeouts, do not change them. "tcp_keepalives_idle = 120", diff --git a/fabfile/pgbench_confs/pgbench_custom.ini b/fabfile/pgbench_confs/pgbench_custom.ini index a2c3dcff..9d5ce3e8 100644 --- a/fabfile/pgbench_confs/pgbench_custom.ini +++ b/fabfile/pgbench_confs/pgbench_custom.ini @@ -1,5 +1,5 @@ [DEFAULT] -postgres_citus_versions: [('12.1', 'release-9.2'), ('12.1', 'release-9.2')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'main')] shard_counts_replication_factors: [(32, 1)] postgresql_conf: [ "max_wal_size = '50GB'", diff --git a/fabfile/pgbench_confs/pgbench_default.ini b/fabfile/pgbench_confs/pgbench_default.ini index ef782b04..bf63883c 100644 --- a/fabfile/pgbench_confs/pgbench_default.ini +++ b/fabfile/pgbench_confs/pgbench_default.ini @@ -1,5 +1,5 @@ [DEFAULT] -postgres_citus_versions: [('14.3', 'release-11.0')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'main')] shard_counts_replication_factors: [(32, 1)] ; Citus flags can be added below as well. E.g. citus.shard_replication_factor = 2 diff --git a/fabfile/pgbench_confs/pgbench_default_without_transaction.ini b/fabfile/pgbench_confs/pgbench_default_without_transaction.ini index e1311ddb..d0ce9d89 100644 --- a/fabfile/pgbench_confs/pgbench_default_without_transaction.ini +++ b/fabfile/pgbench_confs/pgbench_default_without_transaction.ini @@ -1,5 +1,5 @@ [DEFAULT] -postgres_citus_versions: [('14.3', 'release-11.0')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'main')] shard_counts_replication_factors: [(32, 1)] ; Citus flags can be added below as well. E.g. citus.shard_replication_factor = 2 diff --git a/fabfile/pgbench_confs/pgbench_issue_1799.ini b/fabfile/pgbench_confs/pgbench_issue_1799.ini index 1463b227..a84517f7 100644 --- a/fabfile/pgbench_confs/pgbench_issue_1799.ini +++ b/fabfile/pgbench_confs/pgbench_issue_1799.ini @@ -1,5 +1,5 @@ [DEFAULT] -postgres_citus_versions: [('12.1', 'release-9.2')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'main')] shard_counts_replication_factors: [(32, 1)] postgresql_conf: [ "max_wal_size = '50GB'", diff --git a/fabfile/pgbench_confs/scale_test.ini b/fabfile/pgbench_confs/scale_test.ini index e72eb502..4a29e7bf 100644 --- a/fabfile/pgbench_confs/scale_test.ini +++ b/fabfile/pgbench_confs/scale_test.ini @@ -1,5 +1,5 @@ [DEFAULT] -postgres_citus_versions: [('12.1', 'release-9.2'), ('12.1', 'release-9.2')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'main')] shard_counts_replication_factors: [(32, 1)] postgresql_conf: [ "max_wal_size = '50GB'", diff --git a/fabfile/pgbench_confs/scale_test_100_columns.ini b/fabfile/pgbench_confs/scale_test_100_columns.ini index 0f5228c6..8de160c5 100644 --- a/fabfile/pgbench_confs/scale_test_100_columns.ini +++ b/fabfile/pgbench_confs/scale_test_100_columns.ini @@ -1,5 +1,5 @@ [DEFAULT] -postgres_citus_versions: [('12.1', 'release-9.2'), ('12.1', 'release-9.2')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'main')] shard_counts_replication_factors: [(32, 1)] postgresql_conf: [ "max_wal_size = '50GB'", diff --git a/fabfile/pgbench_confs/scale_test_foreign.ini b/fabfile/pgbench_confs/scale_test_foreign.ini index 8c58e945..7fe451db 100644 --- a/fabfile/pgbench_confs/scale_test_foreign.ini +++ b/fabfile/pgbench_confs/scale_test_foreign.ini @@ -1,5 +1,5 @@ [DEFAULT] -postgres_citus_versions: [('12.1', 'release-9.2'), ('12.1', 'release-9.2')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'main')] shard_counts_replication_factors: [(32, 1)] postgresql_conf: [ "max_wal_size = '50GB'", diff --git a/fabfile/pgbench_confs/scale_test_no_index.ini b/fabfile/pgbench_confs/scale_test_no_index.ini index e3c0c1ee..443e6bac 100644 --- a/fabfile/pgbench_confs/scale_test_no_index.ini +++ b/fabfile/pgbench_confs/scale_test_no_index.ini @@ -1,5 +1,5 @@ [DEFAULT] -postgres_citus_versions: [('12.1', 'release-9.2'), ('12.1', 'release-9.2')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'main')] shard_counts_replication_factors: [(32, 1)] postgresql_conf: [ "max_wal_size = '50GB'", diff --git a/fabfile/pgbench_confs/scale_test_prepared.ini b/fabfile/pgbench_confs/scale_test_prepared.ini index d8ed9da0..5bb4fe81 100644 --- a/fabfile/pgbench_confs/scale_test_prepared.ini +++ b/fabfile/pgbench_confs/scale_test_prepared.ini @@ -1,5 +1,5 @@ [DEFAULT] -postgres_citus_versions: [('12.1', 'release-9.2'), ('12.1', 'release-9.2')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'main')] shard_counts_replication_factors: [(32, 1)] postgresql_conf: [ "max_wal_size = '50GB'", diff --git a/fabfile/pgbench_confs/scale_test_reference.ini b/fabfile/pgbench_confs/scale_test_reference.ini index a415a082..3c4c2f39 100644 --- a/fabfile/pgbench_confs/scale_test_reference.ini +++ b/fabfile/pgbench_confs/scale_test_reference.ini @@ -1,5 +1,5 @@ [DEFAULT] -postgres_citus_versions: [('12.1', 'release-9.2'), ('12.1', 'release-9.2')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'main')] shard_counts_replication_factors: [(32, 1)] postgresql_conf: [ "max_wal_size = '50GB'", diff --git a/fabfile/tpch_confs/tpch_default.ini b/fabfile/tpch_confs/tpch_default.ini index c1520842..d93ca7b7 100644 --- a/fabfile/tpch_confs/tpch_default.ini +++ b/fabfile/tpch_confs/tpch_default.ini @@ -1,5 +1,5 @@ [test] -postgres_citus_versions: [('14.3', 'release-11.0')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'main')] tpch_tasks_executor_types: [('1.sql', 'adaptive'), ('3.sql', 'adaptive'), ('5.sql', 'adaptive'), ('6.sql', 'adaptive') ,('7.sql', 'adaptive'), ('8.sql', 'adaptive'), ('9.sql', 'adaptive'), ('10.sql', 'adaptive') , ('12.sql', 'adaptive'), ('14.sql', 'adaptive'), ('19.sql', 'adaptive')] diff --git a/fabfile/tpch_confs/tpch_q1.ini b/fabfile/tpch_confs/tpch_q1.ini index fad34262..22f1b674 100644 --- a/fabfile/tpch_confs/tpch_q1.ini +++ b/fabfile/tpch_confs/tpch_q1.ini @@ -1,4 +1,4 @@ [test] -postgres_citus_versions: [('12.1', 'release-11.0')] +postgres_citus_versions: [('15.1', 'release-11.1'), ('15.1', 'main')] tpch_tasks_executor_types: [('1.sql', 'real-time')] scale_factor: 1 \ No newline at end of file diff --git a/fabfile/use.py b/fabfile/use.py index ddf81f4e..6f78cc73 100644 --- a/fabfile/use.py +++ b/fabfile/use.py @@ -12,7 +12,7 @@ @task(positional=['version']) def citus(c, version): - 'Choose a citus version. For example: fab use.citus v6.0.1 setup.basic-testing (defaults to master)' + 'Choose a citus version. For example: fab use.citus v11.1.5 setup.basic-testing (defaults to master)' if multi_connections.execute_on_all_nodes_if_no_hosts(c, citus, version): return diff --git a/hammerdb/create-run.sh b/hammerdb/create-run.sh index 700ffcd1..a9e71c87 100755 --- a/hammerdb/create-run.sh +++ b/hammerdb/create-run.sh @@ -10,7 +10,7 @@ set -x is_tpcc=${IS_TPCC:=true} # set to true if you want tpcc to be run, otherwise set to false is_ch=${IS_CH:=false} # set to true if you want ch benchmark to be run, otherwise set to false username=pguser # username of the database -hammerdb_version=4.4 # (4.4+ recommended) find available versions in download-hammerdb.sh +hammerdb_version=4.6 # (4.4+ recommended) find available versions in download-hammerdb.sh # ssh_execute is used to run a command multiple times on ssh, this is because we sometimes get timeouts # while trying to ssh, and it shouldn't make the script fail. If a command actually fails, it will always diff --git a/hammerdb/download-hammerdb.sh b/hammerdb/download-hammerdb.sh index f49cd597..f7241bbe 100755 --- a/hammerdb/download-hammerdb.sh +++ b/hammerdb/download-hammerdb.sh @@ -10,7 +10,17 @@ if test -d "HammerDB-$1"; then exit fi -if [ "$1" == 4.4 ] +if [ "$1" == 4.6 ] +then + SHA1=f1ce8891e24dba9dfa99c16781673f37d71e94df + OUTPUT=HammerDB-4.6-Linux.tar.gz + URL=https://github.com/TPC-Council/HammerDB/releases/download/v4.6/$OUTPUT +elif [ "$1" == 4.5 ] +then + SHA1=98c924ad4e99d6f55ef89f098ce638455473dba6 + OUTPUT=HammerDB-4.5-Linux.tar.gz + URL=https://github.com/TPC-Council/HammerDB/releases/download/v4.5/$OUTPUT +elif [ "$1" == 4.4 ] then SHA1=5347e06baad690336afa6d354330cd59d98343cc OUTPUT=HammerDB-4.4-Linux.tar.gz @@ -31,7 +41,7 @@ then OUTPUT=HammerDB-3.3-Linux.tar.gz URL=https://github.com/TPC-Council/HammerDB/releases/download/v3.3/$OUTPUT else - echo 'Expects version parameter. Supported versions: 3.3 4.0 4.3 4.4' 1>&2 + echo 'Expects version parameter. Supported versions: 3.3 4.0 4.3 4.4 4.5 4.6' 1>&2 exit 1 fi diff --git a/hammerdb/setup.sh b/hammerdb/setup.sh index 8ed7038d..338d5a96 100755 --- a/hammerdb/setup.sh +++ b/hammerdb/setup.sh @@ -21,7 +21,7 @@ mv HammerDB-"${hammerdb_version}" ~/ # postgres is necessary for hammerdb, so install that sudo yum -y install https://download.postgresql.org/pub/repos/yum/reporpms/EL-9-x86_64/pgdg-redhat-repo-latest.noarch.rpm -sudo yum -y install postgresql14-server postgresql14 +sudo yum -y install postgresql15-server postgresql15 cd "${driverdir}" diff --git a/jdbc/jdbc_config.json b/jdbc/jdbc_config.json index 387b34ae..110d7f8c 100644 --- a/jdbc/jdbc_config.json +++ b/jdbc/jdbc_config.json @@ -1,6 +1,6 @@ { - "pg_version": "14.3", - "citus_branch": "release-11.0", + "pg_version": "15.1", + "citus_branch": "release-11.1", "COMMENT_NOT_PARSED": "You can lookup jdbc versions at: https://jdbc.postgresql.org/download.html", "jdbc_version": "42.3.6" } diff --git a/release/README.md b/release/README.md index 1ebf4caa..18b81ac4 100644 --- a/release/README.md +++ b/release/README.md @@ -2,14 +2,14 @@ # Usage of update_files.sh If you will test a custom branch, you can update all the config files with `./update_files.sh`. Say that you -will do the release testing for `release-9.2` and `release-9.3`. You can do the following: +will do the release testing for `release-11.1` and `main`. You can do the following: ```bash vim update_files.sh -new_branch1=release-9.3 -new_branch2=release-9.2 -new_pg_version=12.2 +new_branch1=release-11.1 +new_branch2=main +new_pg_version=15.1 ./update_files.sh # this will update all the branches correctly git diff #make sure that everything is fine diff --git a/release/update_files.sh b/release/update_files.sh index 5bb8ae92..4718404e 100755 --- a/release/update_files.sh +++ b/release/update_files.sh @@ -8,9 +8,9 @@ set -e set -x ## VARIABLES ## -new_branch1=release-9.3 -new_branch2=release-9.2 -new_pg_version=12.2 +new_branch1=release-11.1 +new_branch2=main +new_pg_version=15.1 ## VARIABLES ## releasedir="${0%/*}"