f4278c45ef
* Maintenance Window: Introduced dynamic update to polling time. (#245) Resolves #245 Added new configuration parameters and controller handling that can be used to enable reduction in polling time near to start of a maintenance window. - The device may only get to know of start of maintenance window when the device polls the server. If a window is available, server responds with maintenanceWindow as 'available' and the update as 'forced'. Hence, as the time for start of a maintenance window approaches, the device should poll the server more often in order to decrease the possibility of missing the maintenance window start time due to non-alignment of polling with the start time, or due to packet losses and other network issues. - Additional configuration parameter is introduced that will be used to specify the minimum number of times the controller should poll between current time and the start of maintenance window. Using this parameter, an "inverse exponential back-off" scheme for reducing the polling time is implemented. - Implemented inverse exponential back-off for polling. For example, considering default polling-time = 00:05:00, min-polling-time = 00:00:30, maintenance-window-poll-count = 3, we expect that controller should poll the server 3 times between current time and start of a maintenance window. Thus in following scenario controller will get to know of start of maintenance 3 secs after scheduled start. Whereas without this scheme, delay may be as much as 4 minutes. time until start: 00:21:00; polling time: 00:05:00 time until start: 00:16:00; polling time: 00:05:00 time until start: 00:11:00; polling time: 00:03:40 time until start: 00:07:20; polling time: 00:02:27 time until start: 00:04:53; polling time: 00:01:38 time until start: 00:02:10; polling time: 00:00:43 time until start: 00:01:27; polling time: 00:00:30 time until start: 00:00:57; polling time: 00:00:30 time until start: 00:00:27; polling time: 00:00:30 time until start: -00:00:03; polling time: 00:05:00 - Once a maintenance window starts, the polling time is reverted to default. Polling time is not changed near the end of a maintenance window. Signed-off-by: Christian Storm <christian.storm@siemens.com> Signed-off-by: Himanshu Kumar Singh <himanshu.singh@siemens.com> Signed-off-by: Raju HS <raju.hs@siemens.com> * Maintenance Window: Updated the DMF API. Added maintenance window handling for DMF API. Currently no timer or scheduler is used to send updates to devices when a maintenance window becomes available. The device can simulate a poll behaviour by sending a message on UPDATE_ACTION_STATUS topic. Depending on whether there is a maintenance window available or not, server should send DOWNLOAD_AND_INSTALL or DOWNLOAD_AND_SKIP. - Additional action status (DOWNLOADED) and event topic (DOWNLOAD_AND_SKIP) - Server sends DOWNLOAD_AND_SKIP or DOWNLOAD_AND_INSTALL based on maintenance window using ControllerManagement Signed-off-by: Christian Storm <christian.storm@siemens.com> Signed-off-by: Himanshu Kumar Singh <himanshu.singh@siemens.com> Signed-off-by: Raju HS <raju.hs@siemens.com> * Maintenance Window: Added tests for DDI and Management APIs. Added test cases for DDI and Management API to test for maintenance window related logic. - Tests for Management API for DistributionSet assignment with and without valid maintenance schedule specified. - Tests for lapsed maintenance schedule. - Additional test helper methods. Signed-off-by: Christian Storm <christian.storm@siemens.com> Signed-off-by: Himanshu Kumar Singh <himanshu.singh@siemens.com> Signed-off-by: Raju HS <raju.hs@siemens.com> * Maintenance Window: Added new fields, enums, APIs and helper classes. Extended the data model and Action API to support maintenance schedule. Also provided helper classes to parse cron expression and calculate maintenance window based on the schedule. - DOWNLOADED status added to Action interface for controller to record action status. - Extra fields and APIs are added to Action to set schedule, duration and time zone of maintenance window. - Maintenance schedule can be defined using: a cron expression (e.g. "0 30 10-13 ? * WED,FRI" for 30 minutes past the hour, between 10:00 AM and 01:59 PM, only on Wednesday and Friday), a duration (e.g. "00:30:00" for 30 minutes) and the time zone (e.g. "+02:00" for CET summer time or "+00:00" for UTC) relative to which maintenance window start time is calculated. - Extra APIs added to check if a maintenance schedule is defined or lapsed and if a maintenance window is available based on the schedule. - Added database fields to store the schedule, duration and timezone of maintenance window. - Added helper classes to parse and calculate the maintenance window schedule. - Added exception handling classes to handle invalid maintenance window schedule. Signed-off-by: Christian Storm <christian.storm@siemens.com> Signed-off-by: Himanshu Kumar Singh <himanshu.singh@siemens.com> Signed-off-by: Raju HS <raju.hs@siemens.com> * Maintenance Window: Extended JSON model for REST APIs. Extended the JSON model to support defining and communicating the maintenance window information using the REST APIs. - Additional JSON objects are defined to extend request and response bodies. - When requesting a distribution set assignment to a target, the maintenance schedule information can be specified. Example below specifies a maintenance schedule at 11:00 PM, only on Saturday, only in 2017, for 30 minutes. Maintenance window start time is then calculated relative to the time zone specified, in this case CET summer time. $ curl 'http://127.0.0.1:8080/rest/v1/targets/2/assignedDS' -i -X POST -H 'Content-Type: application/json;charset=UTF-8' -d '{ "forcetime" : "0", "id" : "1", "type" : "forced", "maintenanceWindow": { "schedule": "0 0 23 ? * SAT 2017", "duration": "00:30:00", "timezone": "+02:00" } }' - When device requests the action information it gets to know if there is a 'maintenance schedule' for this action or not based on presence of maintenanceWindow field in the response. The value of maintenanceWindow is either 'available' (i.e. the maintenance window is now available as per defined schedule and the update can progress) or 'unavailable' (implying that maintenance window is not available now and update should not be attempted). If there is no 'maintenance schedule' defined, the field is null or not present. $ curl 'http://127.0.0.1:8080/default/controller/v1/2/deploymentBase/1?c=411629670' -i -H 'Accept: application/hal+json' HTTP/1.1 200 OK Content-Type: application/hal+json;charset=UTF-8 Content-Length: 1665 { "id": "1", "deployment": { "download": "forced", "update": "forced", "chunks": [ { "part": "os", "version": "1", "name": "1", "artifacts" : [ { "filename" : "binary.tgz", "hashes" : { "sha1" : "8b71973e835cd0718827238b1cf89f0079e44dae", "md5" : "a90218782025c48e09ba0436cf30d029" }, "size" : 5, "_links" : { "download" : { "href" : "http://127.0.0.1:8080/api/v1/default/download/controller/2/softwaremodules/8/filename/binary.tgz" }, "md5sum" : { "href" : "http://127.0.0.1:8080/api/v1/default/download/controller/2/softwaremodules/8/filename/binary.tgz.MD5SUM" }, "download-http" : { "href" : "http://127.0.0.1:8080/api/v1/default/download/controller/2/softwaremodules/8/filename/binary.tgz" }, "md5sum-http" : { "href" : "http://127.0.0.1:8080/api/v1/default/download/controller/2/softwaremodules/8/filename/binary.tgz.MD5SUM" } } } ] } ], "maintenanceWindow": "available" } } Signed-off-by: Christian Storm <christian.storm@siemens.com> Signed-off-by: Himanshu Kumar Singh <himanshu.singh@siemens.com> Signed-off-by: Raju HS <raju.hs@siemens.com> * Maintenance Window: Added additional UI elements. Added additional UI elements to specify maintenance window schedule in hawkBit UI while assigning distribution set(s) to target(s). - Added UI control to hawkBit UI when distribution set assignment is confirmed. - Also added required theme updates and resource strings. Signed-off-by: Christian Storm <christian.storm@siemens.com> Signed-off-by: Himanshu Kumar Singh <himanshu.singh@siemens.com> Signed-off-by: Raju HS <raju.hs@siemens.com> * Maintenance Window: Modified DDI and Mgmt APIs for maintenance window. Modified controller management and deployment management to handle maintenance window schedule in Management API and DDI API. - Modified REST endpoints to accept and respond with maintenance window information. $ curl 'http://127.0.0.1:8080/rest/v1/targets/2/assignedDS' -i -X POST -H 'Content-Type: application/json;charset=UTF-8' -d '{ "forcetime" : "0", "id" : "1", "type" : "forced", "maintenanceWindow": { "schedule": "0 0 23 ? * SAT 2017", "duration": "00:30:00", "timezone": "+02:00" } }' - Check for availability of a maintenance window based on the schedule. - Device gets go-ahead to install by change in ETAG and 'update' status to 'forced' when it polls during an available maintenance window. $ curl 'http://127.0.0.1:8080/default/controller/v1/2/deploymentBase/1?c=411659461' -i -H 'Accept: application/hal+json' HTTP/1.1 200 OK Content-Type: application/hal+json;charset=UTF-8 Content-Length: 1665 { "id": "1", "deployment": { "download": "forced", "update": "forced", "chunks": [ { "part": "os", "version": "1", "name": "1", "artifacts" : [ { "filename" : "binary.tgz", "hashes" : { "sha1" : "8b71973e835cd0718827238b1cf89f0079e44dae", "md5" : "a90218782025c48e09ba0436cf30d029" }, "size" : 5, "_links" : { "download" : { "href" : "http://127.0.0.1:8080/api/v1/default/download/controller/2/softwaremodules/8/filename/binary.tgz" }, "md5sum" : { "href" : "http://127.0.0.1:8080/api/v1/default/download/controller/2/softwaremodules/8/filename/binary.tgz.MD5SUM" }, "download-http" : { "href" : "http://127.0.0.1:8080/api/v1/default/download/controller/2/softwaremodules/8/filename/binary.tgz" }, "md5sum-http" : { "href" : "http://127.0.0.1:8080/api/v1/default/download/controller/2/softwaremodules/8/filename/binary.tgz.MD5SUM" } } } ] } ], "maintenanceWindow": "available" } } - In case the current available maintenance window lapses, the device is again sent 'update' status as 'skip'. The device may choose then to abort the update and restart during next available window. $ curl 'http://127.0.0.1:8080/default/controller/v1/2/deploymentBase/1?c=411659455' -i -H 'Accept: application/hal+json' HTTP/1.1 200 OK Content-Type: application/hal+json;charset=UTF-8 Content-Length: 1665 { "id": "1", "deployment": { "download": "forced", "update": "skip", "chunks": [ { "part": "os", "version": "1", "name": "1", "artifacts" : [ { "filename" : "binary.tgz", "hashes" : { "sha1" : "8b71973e835cd0718827238b1cf89f0079e44dae", "md5" : "a90218782025c48e09ba0436cf30d029" }, "size" : 5, "_links" : { "download" : { "href" : "http://127.0.0.1:8080/api/v1/default/download/controller/2/softwaremodules/8/filename/binary.tgz" }, "md5sum" : { "href" : "http://127.0.0.1:8080/api/v1/default/download/controller/2/softwaremodules/8/filename/binary.tgz.MD5SUM" }, "download-http" : { "href" : "http://127.0.0.1:8080/api/v1/default/download/controller/2/softwaremodules/8/filename/binary.tgz" }, "md5sum-http" : { "href" : "http://127.0.0.1:8080/api/v1/default/download/controller/2/softwaremodules/8/filename/binary.tgz.MD5SUM" } } } ] } ], "maintenanceWindow": "unavailable" } } - In case the entire maintenance schedule lapses, before device is able to send the completed status, the action is canceled. $ curl 'http://127.0.0.1:8080/default/controller/v1/2' -i -H 'Accept: application/hal+json' HTTP/1.1 200 OK Content-Type: application/hal+json;charset=UTF-8 Content-Length: 340 { "config": { "polling": { "sleep": "00:05:00" } }, "_links": { "cancelAction": { "href": "http://127.0.0.1:8080/default/controller/v1/2/cancelAction/1" }, "configData": { "href": "http://127.0.0.1:8080/default/controller/v1/2/configData" } } } Signed-off-by: Christian Storm <christian.storm@siemens.com> Signed-off-by: Himanshu Kumar Singh <himanshu.singh@siemens.com> Signed-off-by: Raju HS <raju.hs@siemens.com>
hawkBit metadata repository
The repository is in charge for managing the meta data of the update server, e.g. provisioning targets, distribution sets, software modules etc.
Build
[indent=0]
$ mvn clean install
Note, in order to build correctly in your IDE, you have to add ./target/generated-sources/apt to your build path.
#Concepts
Rollout
A rollout consists of the distribution set and a target-query-filter which defines the targets to be updated in this rollout. The targets within this filter are split up in configured amount of Deployment Groups at creation time.
When starting a rollout, for all targets within this rollout deployment actions will be created. The deployment actions of the first group will be started immediately all other deployment actions will be scheduled.
Due rollouts might include a large number of targets and deployment group, creation as well as starting a rollout might take some time and therefore the creation and starting of an rollout is executed asynchronously. The creation and starting progress is reflected by the rollout's status attribute
Rollout Creation
The targets reflected by the target-query-filter is interpreted at creation time. Only targets which have been created at that time are included in the rollout. Targets which are created after the rollout creation will not be included. At rollout creation time all necessary deployment groups containing the targets will be created. Dynamically adding targets to a created or running rollout is currently not supported.
Rollout Starting
The rollout is using the same concept based on deployment acionts per target. All necessary deployment acionts will be created at starting point but not all deployment acionts will be set to running. Only the deployment acionts of the first deployment group is set to running, all other actions are created in scheduled state.
If targets having not finished deployment actions at rollout starting time, these action are tried to cancel (state: canceling). Scheduled actions from another rollout are canceled directly on server side because they were never running before and can be safely canceled.
Multiple rollouts can be running at the same time, but if the rollouts effect the same targets they will interfer the targets deployment actions e.g. canceling/cancel already running/scheduled deployment actions.
Deployment Group Condition/Action
Success Condition/Action
A success condition defines when a deployment group is interpreted as success and triggers the success action. E.g. a threshold of successfully updated targets within the group.
The success action is executed if the success_condition is fullfilled. Currently only the success action starting the next following deployment group is available.
Error Condition/Action
A error condition defines when a deployment group is interpreted as failure and triggers the error action. E.g. a threshold of update failures of targets within the group.
The error action is executed if the error condition is fullfilled. Currently only the error action pausing the whole rollout is available, a paused rollout can be resumed by a user.
Rollout Scheduler
The rollout is managed by a scheduler task which is checking for rollouts in running state. This is done atomic by updating the row in the database, so only the rollouts are checked and processed for the current instance to avoid that multiple instances are processing a rollout.
Due the scheduler, it might take some time until a rollouts is processed and switching is state or starting the next deployment group.