Calculates exposure of specified subjects to features in the environment. This agent is designed to be deployed in the HD4 stack - https://github.com/TheWorldAvatar/hd4-stack.
- NAMESPACE (namespace of blazegraph, defaults to kb)
- DATABASE (database name of postgres, defaults to postgres)
Note that the debugging and production image are separate images.
To build the production image:
docker compose buildTo push to the repository:
docker compose pushThe stack manager config for production - https://github.com/TheWorldAvatar/hd4-stack/blob/main/stack-manager/inputs/config/services/exposure-calculation-agent.json.
To build the debugging image:
docker compose -f docker-compose-debug.yml buildThe stack manager config for debugging - https://github.com/TheWorldAvatar/hd4-stack/blob/main/stack-manager/inputs/config/services/exposure-calculation-agent-debug.json, debug port is set to 5678.
To attach using VS code, the following config can be added in launch.json
{
"name": "Python: Attach using Debugpy",
"type": "debugpy",
"request": "attach",
"connect": {
"host": "localhost",
"port": 5678
},
"pathMappings": [
{
"localRoot": "${workspaceFolder}",
"remoteRoot": "/app"
}
]
}Route to call the core function:
POST /calculate_exposure with a JSON payload, payload to have the following keys:
- subject: [IRI(s) of subject]
- exposure: [IRI of exposure dataset, added by stack data uploader]
- calculation: [IRI of calculation instance]
Example curl request:
curl -X POST http://localhost:3838/exposure-calculation-agent/calculate_exposure \
-H "Content-Type: application/json" \
-d '{"subject": "http://subject", "exposure": "http://exposure", "calculation": "http://calculation"}'Value of "subject" can be a list of IRIs (JSON array) if the subject is not a trajectory, e.g.
curl -X POST http://localhost:3838/exposure-calculation-agent/calculate_exposure \
-H "Content-Type: application/json" \
-d '{"subject": ["http://subject1", "http://subject2"], "exposure": "http://exposure", "calculation": "http://calculation"}'This agent uses that stack outgoing federation endpoint https://github.com/TheWorldAvatar/stack/tree/main/stack-manager#outgoing-stack-endpoint, please make sure this endpoint is set up correctly, e.g. being able to query the necessary data from here.
Agent considers two types of subject for exposure calculations: subject with a fixed geometry and subject with a trajectory.
Agent will query for the WKT literal in the following form:
<http://subject> geo:asWKT "POINT(1,2)"^^geo:wktLiteralA PostGIS point time series instantiated using the TimeSeriesClient:
<http://subject> <https://www.theworldavatar.com/kg/ontotimeseries/hasTimeSeries> <http://timeseries>Time series data:
| time | points (WKB in database) |
|---|---|
| 1 | POINT(1 2) |
| 2 | POINT(3 4) |
| 3 | POINT(5 6) |
A trajectory can be accompanied by trip data instantiated by the trip agent (https://github.com/TheWorldAvatar/trip-agent), the trip data shares the same time values as the subject of exposure:
| Time | Point | Trip |
|---|---|---|
| 1 | POINT(1 2) | 1 |
| 2 | POINT(3 4) | 1 |
| 3 | POINT(5 6) | 2 |
| 4 | POINT(7 8) | 2 |
Points to a table in PostGIS, assumed to be uploaded using the stack data uploader, the following triples should be queryable:
<http://dataset> a dcat:Dataset;
dcterms:title 'table_name'.The dataset type (raster or vector) depends on the calculation type
Results instantiated depend on the subject type (trajectory or fixed geometry).
Results are instantiated in the following form in Ontop:
PREFIX derivation: <https://www.theworldavatar.com/kg/ontoderivation/>
PREFIX exposure: <https://www.theworldavatar.com/kg/ontoexposure/>
<http://derivation> a derivation:Derivation;
derivation:isDerivedFrom <http://subject>;
derivation:isDerivedFrom <http://exposure>.
<http://result> a exposure:ExposureResult;
exposure:hasCalculationMethod <http://calculation>;
derivation:belongsTo <http://derivation>;
exposure:hasValue 123;
exposure:hasUnit "[-]".The result instance points to a column in a time series table and it shares the same time series with the trajectory. The trajectory should be instantiated using com.cmclinnovations.stack.clients.timeseries.TimeSeriesRDBClient, time series data will be queried from the stack outgoing federated endpoint.
PREFIX derivation: <https://www.theworldavatar.com/kg/ontoderivation/>
PREFIX exposure: <https://www.theworldavatar.com/kg/ontoexposure/>
<http://derivation> a derivation:Derivation;
derivation:isDerivedFrom <http://subject>;
derivation:isDerivedFrom <http://exposure>.
<http://result> a exposure:ExposureResult;
exposure:hasCalculationMethod <http://calculation>;
exposure:hasUnit "[-]";
derivation:belongsTo <http://derivation>.If no trip data is present, the entire trajectory is considered as a single trip and a single value is calculated. If trip data is present, a value is calculated for each trip. A new result instance is added for each subject - exposure - calculation combination. Final results will look something like the following for data with trips. Note that the same value is repeated over each row within a trip.
| Time | Point | Trip | Result A | Result B |
|---|---|---|---|---|
| 1 | POINT(1 2) | 1 | 1 | 2 |
| 2 | POINT(3 4) | 1 | 1 | 2 |
| 3 | POINT(5 6) | 2 | 3 | 4 |
| 4 | POINT(7 8) | 2 | 3 | 4 |
If trip data is not present, entire trajectory is treated as a single trip:
| Time | Point | Result A | Result B |
|---|---|---|---|
| 1 | POINT(1 2) | 1 | 4 |
| 2 | POINT(3 4) | 1 | 4 |
| 3 | POINT(5 6) | 1 | 4 |
| 4 | POINT(5 6) | 1 | 4 |
Supported calculation types:
<https://www.theworldavatar.com/kg/ontoexposure/TrajectoryCount><https://www.theworldavatar.com/kg/ontoexposure/TrajectoryArea><https://www.theworldavatar.com/kg/ontoexposure/Count><https://www.theworldavatar.com/kg/ontoexposure/Area><https://www.theworldavatar.com/kg/ontoexposure/AreaWeightedSum><https://www.theworldavatar.com/kg/ontoexposure/RasterArea><https://www.theworldavatar.com/kg/ontoexposure/TrajectoryAreaWeightedSum>
Permissible metadata depends on the calculation type. A result instance is instantiated for each subject - exposure - calculation combination.
The choice of projection affects the results greatly. For trajectory based calculations, azimuthal equidistant projection (AEQD) is used, the centroid is calculated from the trajectory's envelope. For calculations involving fixed points, EPSG:3857 is used to keep things simple, in case there are points that are far from each other as the AEQD projection relies on a centroid.
Dataset filters:
Any number of dataset filters can be attached to a calculation instance, for example:
PREFIX exposure: <https://www.theworldavatar.com/kg/ontoexposure/>
<http://calculation> a exposure:Area;
exposure:hasDatasetFilter <http://dataset_filter>.
<http://dataset_filter> exposure:hasFilterColumn "year";
exposure:hasFilterValue 2000.This will add WHERE year=2000 into the SQL queries for the calculations.
Overview: Counts features that are within a specified distance from the trajectory using ST_DWithin. SQL query template here
Requirements for subject and exposure dataset:
- Subject: contains a time series of points
- Exposure: Any vector dataset uploaded via the stack data uploader
The instance of this calculation type:
<http://calculation> a <https://www.theworldavatar.com/kg/ontoexposure/TrajectoryCount>;
<https://www.theworldavatar.com/kg/ontoexposure/hasDistance> 100;
<https://www.theworldavatar.com/kg/ontoexposure/hasUpperbound> 456;
<https://www.theworldavatar.com/kg/ontoexposure/hasLowerbound> 123.Distance is mandatory, whereas upper and lower bounds are optional.
Overview: Counts features that are near each subject using ST_DWithin. SQL query template here
Requirements:
Subject: Any fixed vector with a WKT literal associated via geo:asWKT. Exposure: Any vector dataset uploaded via the stack data uploader.
The instance of this calculation type:
<http://calculation> a <https://www.theworldavatar.com/kg/ontoexposure/Count>;
<https://www.theworldavatar.com/kg/ontoexposure/hasDistance> 100.Exposure dataset, optional to specify a custom geometry column name, if it is not present, it will default to 'wkb_geometry'.
<http://exposure> <https://www.theworldavatar.com/kg/ontoexposure/hasGeometryColumn> "wkb_geometry".Overview: Calculates intersected area between the buffered trajectory and specified dataset. SQL query template here
Requirements:
Subject: Point time series Exposure: A polygon dataset
<http://calculation> a <https://www.theworldavatar.com/kg/ontoexposure/TrajectoryArea>;
<https://www.theworldavatar.com/kg/ontoexposure/hasDistance> 100;
<https://www.theworldavatar.com/kg/ontoexposure/hasUpperbound> 456;
<https://www.theworldavatar.com/kg/ontoexposure/hasLowerbound> 123.Distance is mandatory, whereas upper and lower bounds are optional.
Exposure dataset, optional to specify a custom geometry column name, if it is not present, it will default to 'wkb_geometry'.
<http://exposure> <https://www.theworldavatar.com/kg/ontoexposure/hasGeometryColumn> "wkb_geometry".Overview: Calculates intersected area between a buffered point and polygons in a specified dataset. SQL query template here
Requirements:
Subject: Any fixed vector with a WKT literal associated via geo:asWKT, can be a list of IRI Exposure: A polygon dataset
<http://calculation> a <https://www.theworldavatar.com/kg/ontoexposure/Area>;
<https://www.theworldavatar.com/kg/ontoexposure/hasDistance> 100.Exposure dataset, optional to specify a custom geometry column name, if it is not present, it will default to 'wkb_geometry'.
<http://exposure> <https://www.theworldavatar.com/kg/ontoexposure/hasGeometryColumn> "wkb_geometry".Overview: Applies a buffer around a subject and find the intersected pixels in the exposure dataset. Sums up the product of area of the pixel and the value associated with the pixel.
Requirements:
Subject: Any vector with a WKT literal associated via geo:asWKT, can be a list of IRI Exposure: A raster dataset with an area attached to each tile
The following shows the equation:
where
Calculation instance:
<http://calculation> a <https://www.theworldavatar.com/kg/ontoexposure/AreaWeightedSum>;
<https://www.theworldavatar.com/kg/ontoexposure/hasDistance> 100.Exposure dataset, needs to have the area column specified, if geometry column is not specified, it will default to 'rast'.
<http://exposure> a <https://www.theworldavatar.com/kg/ontoexposure/AreaWeightedDataset>;
<https://www.theworldavatar.com/kg/ontoexposure/hasAreaColumn> "area";
<https://www.theworldavatar.com/kg/ontoexposure/hasGeometryColumn> "rast".Overview: Calculates the area enclosed by the buffer. Dataset is expected to have a precalculated area column for each raster tile, as area calculation is not possible on pure raster datasets.
Requirements:
Subject: Any vector with a WKT literal associated via geo:asWKT, can be a list of IRI Exposure: A raster dataset with an area attached to each tile
Calculation instance:
<http://calculation> a <https://www.theworldavatar.com/kg/ontoexposure/RasterArea>;
<https://www.theworldavatar.com/kg/ontoexposure/hasDistance> 400.Exposure dataset, needs to have the area column specified, if geometry column is not specified, it will default to 'rast'.
<http://exposure> a <https://www.theworldavatar.com/kg/ontoexposure/AreaWeightedDataset>;
<https://www.theworldavatar.com/kg/ontoexposure/hasAreaColumn> "area";
<https://www.theworldavatar.com/kg/ontoexposure/hasGeometryColumn> "rast".Trajectory area weighted sum (<https://www.theworldavatar.com/kg/ontoexposure/TrajectoryAreaWeightedSum>)
Similar to <https://www.theworldavatar.com/kg/ontoexposure/AreaWeightedSum>, but for trajectories. Requirements are the same, except that the subject should be a point time series. SQL query template here.
Calculation instance:
<http://calculation> a <https://www.theworldavatar.com/kg/ontoexposure/TrajectoryAreaWeightedSum>;
<https://www.theworldavatar.com/kg/ontoexposure/hasDistance> 100.Exposure dataset, needs to have the area and value columns specified, if geometry column is not specified, it will default to 'wkb_geometry'.
<http://exposure> a <https://www.theworldavatar.com/kg/ontoexposure/AreaWeightedDataset>;
<https://www.theworldavatar.com/kg/ontoexposure/hasAreaColumn> "area";
<https://www.theworldavatar.com/kg/ontoexposure/hasValueColumn> "val";
<https://www.theworldavatar.com/kg/ontoexposure/hasGeometryColumn> "wkb_geometry".These APIs are not part of the core calculation agent and they are located in agent/interactor.
The following APIs are used to initialise the necessary instances and trigger the core agent.
-
/trigger_calculation/ (POST)
Parameters:
- subject_query_file: SPARQL query template to obtain subject IRIs, bind mounted in the folder called
/app/queries. The result of this query should give IRI(s) of subject that we wish to calculate for. The query must have one SELECT parameter and can take any name. - subject: IRI of subject to calculate for
- rdf_type: RDF type of calculation to perform
- distance: buffer distance for calculation
- exposure_table: table name of exposure dataset (needs to be added via the stack data uploader to ensure the necessary triples are present)
- upperbound (optional): optional upperbound for trajectory calculations
- lowerbound (optional): optional lowerbound for trajectory calculations
Either
subject_query_fileorsubjectneeds to be provided in the request. Example usage:curl -X POST http://localhost:3838/exposure-calculation-agent/trigger_calculation/?subject_query_file=subject_query.sparql&rdf_type=https://www.theworldavatar.com/kg/ontoexposure/Count&distance=400&exposure_table=parks
Overview:
- This route checks whether a calculation instance with the specified RDF type and metadata (e.g. distance) exists, then instantiate one if necessary.
- If
subject_query_fileis given, it will run the query to obtain the subject IRIs, otherwise IRI is simply obtained from thesubjectparameter. - Then it queries the dataset IRI of the given
exposure_table, because the core agent is designed to read in IRIs only. - Finally sends a request to the core agent with the IRIs of subject, exposure, and calculation.
- subject_query_file: SPARQL query template to obtain subject IRIs, bind mounted in the folder called
-
/csv_export/non_trajectory (POST)
No parameters, instead inputs to be provided in the request body as JSON, e.g.
curl -X POST "http://localhost:3838/exposure-calculation-agent/csv_export/non_trajectory" -H "Content-Type: application/json" -d @body.json
where the content of body.json is something like
{ "exposure_table": "ndvi_raster", "rdf_type": "https://www.theworldavatar.com/kg/ontoexposure/AreaWeightedSum", "dataset_filter_values": { "year": [ 2016 ] }, // provide either subject_query_file or subject, not both "subject_query_file": "subject_query.sparql", "subject_label_query_file": "subject_label_query.sparql" }Response is a csv file.
-
/csv_export/trajectory (GET)
Parameters:
subject: IRI of subjectrdf_type: RDF type of calculationexposure_table: table name of exposure datasetlowerbound(optional): lowerbound of trajectory time seriesupperbound(optional): upperbound of trajectory time seriesinclude_lat_lng(optional, default true): include geolocation in exported CSV filerefresh_of_cache(optional, default false): refresh outgoing federation cache before querying
Example usage:
curl -o trajectory_result.csv 'http://localhost:3838/exposure-calculation-agent/csv_export/trajectory?rdf_type=https://www.theworldavatar.com/kg/ontoexposure/TrajectoryArea&subject=http://trip_trajectory&exposure_table=parks_2016&lowerbound=1715759710072&upperbound=1715759730231' -
/trigger_calculation/bulk (POST)
No parameters, instead inputs to be provided in the request body as JSON, e.g.
curl -X POST "http://localhost:3838/exposure-calculation-agent/trigger_calculation/bulk" -H "Content-Type: application/json" -d @body.json
where the content of body.json is something like
{ "exposure_table": "ndvi_raster", "rdf_types": [ "https://www.theworldavatar.com/kg/ontoexposure/AreaWeightedSum" ], "distances": [ 400, 800, 1000 ], "dataset_filter_values": { "year": [ 2016 ] }, // provide either subject_query_file or subject, not both "subject_query_file": "subject_query.sparql", "subject": "http://subject" }dataset_filter_values is optional. A cross product between the provided distances and dataset filters is done to produce all combinations of parameters, then calculations are executed for each of the combinations.