A small dataset can "shadow" a large one via a transitive dependency of a calibrated model

[psyhtest]

Since a calibrated image classification model depends on (a subset of) the ImageNet validation set, this dependency can stick and prevent overriding it via program dependency tags (e.g. --dep_add_tags.dataset-imagenet=original).

[psyhtest]

To check that this issue is gone, I'll calibrate a model using a small dataset, and then try to run with a large one.

[psyhtest]

I have the full dataset with the following unique tags:

$ ck show env --tags=imagenet,val,32bits
Env UID:         Target OS: Bits: Name:                         Version: Tags:

4149a4a0cc905062   linux-64    64 ImageNet dataset (validation) 2012     2012,32bits,64bits,caffe,dataset,host-os-linux-32,host-os-linux-64,ilsvrc2012,image-classification,imagenet,jpg,original,raw,target-os-linux-32,target-os-linux-64,v2012,val

I will use them below to direct CK to use it.

$ ck install package --tags=dataset,imagenet,cal,first.500 --dep_add_tags.imagenet-val=min,non-resized
$ ck install package --tags=model,openvino,mobilenet --dep_add_tags.imagenet-cal=first.500
$ ck compile program:mlperf-inference-v0.5
$ export NPROCS=`grep -c processor /proc/cpuinfo`
$ ck run program:mlperf-inference-v0.5 --skip_print_timers \
--cmd_key=image-classification --env.CK_OPENVINO_MODEL_NAME=mobilenet \
--env.CK_LOADGEN_MODE=Accuracy --env.CK_LOADGEN_SCENARIO=Offline \
--env.CK_OPENVINO_NIREQ=$NPROCS --env.CK_OPENVINO_NTHREADS=$NPROCS \
--env.CK_OPENVINO_NSTREAMS=$NPROCS --env.CK_LOADGEN_DATASET_SIZE=500 \
--dep_add_tags.dataset-imagenet=imagenet,val,32bits