Proposal of new diffJobSettings.py functionalities #19

m-dati · 2022-12-20T13:37:19Z

Expanded diff logic with common-extra print,
help improved,
short options added, -j, -n, -c
json comments display added
single ID print added, to display all #ID content [comments optionally included]

$diffJobSettings.py
usage: diffJobSettings [-h] [--host HOST] [--jobids JOBIDS] [--nossl] [--comments]

openqa test data:display the differences of Settings betwen 2 given tests IDs Or display the full data set of a single test ID

options:
  -h, --help            show this help message and exit
  --host HOST           default openqa.suse.de
  --jobids JOBIDS, -j JOBIDS
                        jobid1[,jobid2] # Enter 1[Full data] or 2[Differences] IDs
  --nossl, -n           http urls; default https
  --comments, -c        print job comments too

m-dati · 2022-12-20T13:39:09Z

Example run :

diffJobSettings.py -j 10193771,10215155 -c
Result url:https://openqa.suse.de/api/v1/jobs/ - jobids:['10193771', '10215155']

*** All common keys & values 
['10193771', '10215155'] {
  "ARCH": "x86_64",
  "AZURE_LOGIN_WAIT_SECONDS": "600",
  "BACKEND": "qemu",
  "BETA": "0",
  "BOOT_HDD_IMAGE": "1",
  "BUILD": "0127",
  "DESKTOP": "textmode",
  "DISTRI": "sle",
  "FLAVOR": "Azure-BYOS",
  "HDD_1": "publiccloud_tools_0049.qcow2",
  "INSTALLONLY": "1",
  "KEEP_GRUB_TIMEOUT": "1",
  "LTP_KNOWN_ISSUES": "http://qam.suse.de/media/downloads/ltp_known_issues.json",
  "LTP_TIMEOUT": "10800",
  "MACHINE": "az_Standard_B2s",
  "NOVIDEO": "1",
  "PUBLIC_CLOUD": "1",
  "PUBLIC_CLOUD_BUILD": "1.60",
  "PUBLIC_CLOUD_BUILD_KIWI": "0.9.0",
  "PUBLIC_CLOUD_CREDENTIALS_URL": "https://publiccloud-ng.qa.suse.de/secrets/",
  "PUBLIC_CLOUD_DEFAULT_TTL": "15000",
  "PUBLIC_CLOUD_IMAGE_LOCATION": "http://dist.suse.de/ibs/SUSE:/SLE-15-SP5:/GA:/TEST/images/SLES15-SP5-BYOS.x86_64-0.9.0-Azure-Build1.60.vhdfixed.xz",
  "PUBLIC_CLOUD_INSTANCE_TYPE": "Standard_B2s",
  "PUBLIC_CLOUD_LTP": "1",
  "PUBLIC_CLOUD_NAMESPACE": "ccoe-qac",
  "PUBLIC_CLOUD_PROVIDER": "AZURE",
  "PUBLIC_CLOUD_RESOURCE_NAME": "openqa-suse-de",
  "PUBLIC_CLOUD_RETRY_LOOP": "1",
  "PUBLIC_CLOUD_SCC_ENDPOINT": "SUSEConnect",
  "QEMU_DISABLE_SNAPSHOTS": "1",
  "SCC_REGCODE": "<reg-code-hidden>",
  "START_AFTER_TEST": "publiccloud_upload_img@64bit",
  "TERRAFORM_TIMEOUT": "1620",
  "TEST_SUITE_NAME": "-",
  "VERSION": "15-SP5",
  "VIDEOMODE": "text",
  "VIRTIO_CONSOLE": "1",
  "WORKER_CLASS": "pc_azure"
}

*** Common keys, different values:
10193771 {
  "LTP_COMMAND_FILE": "cve",
  "NAME": "10193771-sle-15-SP5-Azure-BYOS-x86_64-Build0127-publiccloud_ltp_cve@az_Standard_B2s",
  "TEST": "publiccloud_ltp_cve"
}
10215155 {
  "LTP_COMMAND_FILE": "syscalls",
  "NAME": "10215155-sle-15-SP5-Azure-BYOS-x86_64-Build0127-publiccloud_ltp_syscalls@az_Standard_B2s",
  "TEST": "publiccloud_ltp_syscalls"
}

*** Extra keys:
10193771 {}
10215155 {
  "LTP_COMMAND_EXCLUDE": "^(getrusage04|writev03|msgstress04|inotify11)",
  "MAX_JOB_TIME": "43200"
}

*** Comments:
10193771 []
10215155 [
  {
    "bugrefs": [
      "poo#120288"
    ],
    "created": "2022-12-20 13:09:59 +0000",
    "id": 712465,
    "renderedMarkdown": "<p><a href=\"https://progress.opensuse.org/issues/120288\">poo#120288</a> [tools] cloud based tests fail due to traffic to cloud blocked auto_review:&quot;2022-11-0.*Test died: (Waiting for Godot.*ssh|Cannot find image after upload)&quot;:retry</p>\n<p>(Automatic takeover from <a href=\"/tests/9908669\">t#9908669</a>)</p>\n",
    "text": "poo#120288 [tools] cloud based tests fail due to traffic to cloud blocked auto_review:\"2022-11-0.*Test died: (Waiting for Godot.*ssh|Cannot find image after upload)\":retry\n\n(Automatic takeover from t#9908669)\n",
    "updated": "2022-12-20 13:09:59 +0000",
    "userName": "okurz"
  },
  {
    "bugrefs": [],
    "created": "2022-12-20 13:14:31 +0000",
    "id": 712469,
    "renderedMarkdown": "<p>Code: ResourceNotFound\nMessage: The Resource 'Microsoft.Compute/images/SLES15-SP5-BYOS.x86_64-0.9.0-Azure-Build1.60.vhd' under resource group 'openqa-upload' was not found</p>\n",
    "text": "Code: ResourceNotFound\r\nMessage: The Resource 'Microsoft.Compute/images/SLES15-SP5-BYOS.x86_64-0.9.0-Azure-Build1.60.vhd' under resource group 'openqa-upload' was not found",
    "updated": "2022-12-20 13:14:31 +0000",
    "userName": "mdati"
  }
]

m-dati · 2022-12-20T13:46:54Z

Single ID run example:

diffJobSettings.py --jobids 10193771
Result url:https://openqa.suse.de/api/v1/jobs/ - jobids:['10193771']

 {
  "job": {
    "assets": {
      "hdd": [
        "publiccloud_tools_0049.qcow2"
      ]
    },
    "assigned_worker_id": 2191,
    "blocked_by_id": null,
    "children": {
      "Chained": [],
      "Directly chained": [],
      "Parallel": []
    },
    "clone_id": null,
    "group": "Latest - Azure",
    "group_id": 219,
    "has_parents": 1,
    "id": 10193771,
    "name": "sle-15-SP5-Azure-BYOS-x86_64-Build0127-publiccloud_ltp_cve@az_Standard_B2s",
    "origin_id": 10187998,
    "parent_group": "Public Cloud",
    "parent_group_id": 27,
    "parents": {
      "Chained": [
        10187991
      ],
      "Directly chained": [],
      "Parallel": []
    },
    "parents_ok": 1,
    "priority": 50,
    "result": "passed",
    "settings": {
      "ARCH": "x86_64",
      "AZURE_LOGIN_WAIT_SECONDS": "600",
      "BACKEND": "qemu",
      "BETA": "0",
      "BOOT_HDD_IMAGE": "1",
      "BUILD": "0127",
      "DESKTOP": "textmode",
      "DISTRI": "sle",
      "FLAVOR": "Azure-BYOS",
      "HDD_1": "publiccloud_tools_0049.qcow2",
      "INSTALLONLY": "1",
      "KEEP_GRUB_TIMEOUT": "1",
      "LTP_COMMAND_FILE": "cve",
      "LTP_KNOWN_ISSUES": "http://qam.suse.de/media/downloads/ltp_known_issues.json",
      "LTP_TIMEOUT": "10800",
      "MACHINE": "az_Standard_B2s",
      "NAME": "10193771-sle-15-SP5-Azure-BYOS-x86_64-Build0127-publiccloud_ltp_cve@az_Standard_B2s",
      "NOVIDEO": "1",
      "PUBLIC_CLOUD": "1",
      "PUBLIC_CLOUD_BUILD": "1.60",
      "PUBLIC_CLOUD_BUILD_KIWI": "0.9.0",
      "PUBLIC_CLOUD_CREDENTIALS_URL": "https://publiccloud-ng.qa.suse.de/secrets/",
      "PUBLIC_CLOUD_DEFAULT_TTL": "15000",
      "PUBLIC_CLOUD_IMAGE_LOCATION": "http://dist.suse.de/ibs/SUSE:/SLE-15-SP5:/GA:/TEST/images/SLES15-SP5-BYOS.x86_64-0.9.0-Azure-Build1.60.vhdfixed.xz",
      "PUBLIC_CLOUD_INSTANCE_TYPE": "Standard_B2s",
      "PUBLIC_CLOUD_LTP": "1",
      "PUBLIC_CLOUD_NAMESPACE": "ccoe-qac",
      "PUBLIC_CLOUD_PROVIDER": "AZURE",
      "PUBLIC_CLOUD_RESOURCE_NAME": "openqa-suse-de",
      "PUBLIC_CLOUD_RETRY_LOOP": "1",
      "PUBLIC_CLOUD_SCC_ENDPOINT": "SUSEConnect",
      "QEMU_DISABLE_SNAPSHOTS": "1",
      "SCC_REGCODE": "<regcode-hidden>",
      "START_AFTER_TEST": "publiccloud_upload_img@64bit",
      "TERRAFORM_TIMEOUT": "1620",
      "TEST": "publiccloud_ltp_cve",
      "TEST_SUITE_NAME": "-",
      "VERSION": "15-SP5",
      "VIDEOMODE": "text",
      "VIRTIO_CONSOLE": "1",
      "WORKER_CLASS": "pc_azure"
    },
    "state": "done",
    "t_finished": "2022-12-16T10:39:03",
    "t_started": "2022-12-16T09:59:59",
    "test": "publiccloud_ltp_cve"
  }
}

diffJobSettings.py

asmorodskyi · 2022-12-20T13:58:03Z

diffJobSettings.py

 args = parser.parse_args()

 source_url=''
-obvious_differs=("NAME")


why you have dropped this variable ?

As I understood, it was needed to exclude NAME from difference list (continue).
In this update I included the NAME parameter too in the list of data, to show all the differences, therefore this variable and related check-if was no more needed.

But eventually, for more local customizations, I could restore both variable and check and leave empty that list: obvious_differs=()

skip-list applied

when I was using first versions of this script I have seen a lot of variables which always different and knowing that they different does not bring any valuable info . For example there is "WORKER_ID" variable ofc. it will differ from run to run because certain worker picked up randomly from currently available and it does not help you anyhow knowing that yet another random value is selected . Same goes to test "NAME" , it will be always different because NAME variable contains job ID so name is unique .
Main purpose of this script ( at least for me ) is to compare failed job with same one but passed, to try to find single one which cause the failure . The purpose of "obvious_differes" variables is to remove the "noise" from the list . To see only variables which really have chance to be failure cause and NAME have almost no chance to be that variable ( as many others , I planned to extend this list but never had a chance ... )

I agree with your POV. I restored the code, but I let empty the list: in this case, I gonna restore the original settings (NAME), so that each one local copy can be eventually changed on one own need.
Future implementations could eventually provide a more option with fileds to skip :-)

I don't see this changes yet , maybe your forgot to push them ?

changes ongoing ... :) will be online soon

Please, see the last commit: added skip default 'NAME' and a new option to eventually enter more fileds to skip.

Consider we that the job json has up to 3 levels (job.level1.level2) dictionary and actual diff is only performed in the job.settings.. Therefore the current excluded fields can be in that 3rd level only, none in other 2nd levels as job.assets or job.state, in this implementation :)

asmorodskyi · 2022-12-20T13:59:27Z

generally I like it , and I gonna merge it . well done Maurizio ! But let's clarify few small questions

- Expanded diff logic with common-extra print, - help improved, - short options added, - json comments display added - single ID print added

m-dati · 2022-12-20T14:48:24Z

In a future version the 'sensible' data could be optionally auto-hidden

m-dati · 2022-12-21T11:06:09Z

In this last commit
restored default 'NAME' filed to skip
added new option -s to eventually enter a new desired comma-separated list of fileds to skip,
added try checks to handle errors

- skipdiff opt added: list of settings fields to skip from all diffs; default NAME

asmorodskyi reviewed Dec 20, 2022

View reviewed changes

diffJobSettings.py Outdated Show resolved Hide resolved

asmorodskyi reviewed Dec 20, 2022

View reviewed changes

diffJobSettings.py Outdated Show resolved Hide resolved

asmorodskyi reviewed Dec 20, 2022

View reviewed changes

Proposal of new diffJobSettings.py functionality

80d9b24

- Expanded diff logic with common-extra print, - help improved, - short options added, - json comments display added - single ID print added

m-dati force-pushed the diff_update branch from 46b6055 to 80d9b24 Compare December 20, 2022 14:35

m-dati force-pushed the diff_update branch from cfff8fe to ceb1e75 Compare December 21, 2022 10:56

Proposal of additional diffJobSettings functionality

7c855b4

- skipdiff opt added: list of settings fields to skip from all diffs; default NAME

m-dati force-pushed the diff_update branch from ceb1e75 to 7c855b4 Compare December 21, 2022 11:32

Proposal of new diffJobSettings.py functionalities #19

Are you sure you want to change the base?

Proposal of new diffJobSettings.py functionalities #19

Uh oh!

Conversation

m-dati commented Dec 20, 2022

Uh oh!

m-dati commented Dec 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

m-dati commented Dec 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

m-dati Dec 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

m-dati Dec 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

asmorodskyi commented Dec 20, 2022

Uh oh!

m-dati commented Dec 20, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

m-dati commented Dec 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

m-dati commented Dec 20, 2022 •

edited

Loading

m-dati commented Dec 20, 2022 •

edited

Loading

m-dati Dec 20, 2022 •

edited

Loading

m-dati Dec 21, 2022 •

edited

Loading

m-dati commented Dec 20, 2022 •

edited

Loading

m-dati commented Dec 21, 2022 •

edited

Loading