Protection against misuse with system-wide quota definitions (#670)

* Added quota for meta data per software module

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Added unit test for "meta data per module" quota

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* quota test enhancements

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Verify enforcement of meta data quota via REST

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quota for distribution set meta data

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Verify enforcement of distribution set meta data quota via REST

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* software modules per distribution set quota

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Integration test enhancements for Modules per DistSet quota

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quota for software module types per distribution set type

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quota for max artifacts per software module

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quotas for ActionStatus per Action and Messages per ActionStatus

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quota attributes per target

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quota targets per rollout group

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quota max targets per rollout group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per rollout group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per rollout group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per rollout group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* quota max actions per target

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per rollout group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max actions per target

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max actions per target

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per auto assignment

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per manual assignment

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per auto assignment

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per auto assign

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per auto assignment

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max actions per target

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per manual assignment

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix issues caused by merge

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Improve JavaDoc

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Sonar issues

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Sonar findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max artifact size

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Optimize quota configuration

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix test failures

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Improve test coverage

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max rollout groups per rollout

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Configure Rollout UI enhancements

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* UI enhancements

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Minor changes

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Sonar findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Sonar findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Sonar finding

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix code review findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix review findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* New approach for 'max artifact size' enforcement

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Sonar findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Reduce max artifact size for tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Kai's review findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>
This commit is contained in:
Stefan Behl
2018-05-02 12:09:29 +02:00
committed by Kai Zimmermann
parent fcc15a0484
commit 6dd98d2134
63 changed files with 2383 additions and 523 deletions

View File

@@ -14,6 +14,7 @@ import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import org.eclipse.hawkbit.repository.QuotaManagement;
import org.eclipse.hawkbit.repository.RepositoryConstants;
import org.eclipse.hawkbit.repository.event.remote.TargetAssignDistributionSetEvent;
import org.eclipse.hawkbit.repository.event.remote.entity.CancelTargetAssignmentEvent;
@@ -24,6 +25,7 @@ import org.eclipse.hawkbit.repository.jpa.model.JpaAction;
import org.eclipse.hawkbit.repository.jpa.model.JpaActionStatus;
import org.eclipse.hawkbit.repository.jpa.model.JpaDistributionSet;
import org.eclipse.hawkbit.repository.jpa.model.JpaTarget;
import org.eclipse.hawkbit.repository.jpa.utils.QuotaHelper;
import org.eclipse.hawkbit.repository.model.Action;
import org.eclipse.hawkbit.repository.model.Action.Status;
import org.eclipse.hawkbit.repository.model.DistributionSet;
@@ -46,17 +48,19 @@ public abstract class AbstractDsAssignmentStrategy {
protected final ApplicationContext applicationContext;
private final ActionRepository actionRepository;
private final ActionStatusRepository actionStatusRepository;
private final QuotaManagement quotaManagement;
AbstractDsAssignmentStrategy(final TargetRepository targetRepository,
final AfterTransactionCommitExecutor afterCommit, final ApplicationEventPublisher eventPublisher,
final ApplicationContext applicationContext, final ActionRepository actionRepository,
final ActionStatusRepository actionStatusRepository) {
final ActionStatusRepository actionStatusRepository, final QuotaManagement quotaManagement) {
this.targetRepository = targetRepository;
this.afterCommit = afterCommit;
this.eventPublisher = eventPublisher;
this.applicationContext = applicationContext;
this.actionRepository = actionRepository;
this.actionStatusRepository = actionStatusRepository;
this.quotaManagement = quotaManagement;
}
/**
@@ -213,6 +217,11 @@ public abstract class AbstractDsAssignmentStrategy {
JpaAction createTargetAction(final Map<String, TargetWithActionType> targetsWithActionMap, final JpaTarget target,
final JpaDistributionSet set) {
// enforce the 'max actions per target' quota
assertActionsPerTargetQuota(target, 1);
// create the action
final JpaAction actionForTarget = new JpaAction();
final TargetWithActionType targetWithActionType = targetsWithActionMap.get(target.getControllerId());
actionForTarget.setActionType(targetWithActionType.getActionType());
@@ -237,4 +246,11 @@ public abstract class AbstractDsAssignmentStrategy {
return actionStatus;
}
private void assertActionsPerTargetQuota(final Target target, final int requested) {
final int quota = quotaManagement.getMaxActionsPerTarget();
QuotaHelper.assertAssignmentQuota(target.getId(), requested, quota, Action.class, Target.class,
actionRepository::countByTargetId);
}
}

View File

@@ -39,6 +39,16 @@ public interface ActionStatusRepository
*/
Long countByAction(JpaAction action);
/**
* Counts {@link ActionStatus} entries of given {@link Action} in
* repository.
*
* @param actionId
* of the action to count status entries for
* @return number of actions in repository
*/
long countByActionId(Long actionId);
/**
* Retrieves all {@link ActionStatus} entries from repository of given
* ActionId.

View File

@@ -13,6 +13,7 @@ import org.eclipse.hawkbit.repository.jpa.model.JpaDistributionSetMetadata;
import org.eclipse.hawkbit.repository.model.DistributionSetMetadata;
import org.springframework.data.jpa.repository.JpaSpecificationExecutor;
import org.springframework.data.repository.PagingAndSortingRepository;
import org.springframework.data.repository.query.Param;
import org.springframework.transaction.annotation.Transactional;
/**
@@ -23,4 +24,14 @@ public interface DistributionSetMetadataRepository
extends PagingAndSortingRepository<JpaDistributionSetMetadata, DsMetadataCompositeKey>,
JpaSpecificationExecutor<JpaDistributionSetMetadata> {
/**
* Counts the meta data entries that match the given distribution set ID.
*
* @param id
* of the distribution set.
*
* @return The number of matching meta data entries.
*/
long countByDistributionSetId(@Param("id") Long id);
}

View File

@@ -15,6 +15,7 @@ import javax.persistence.EntityManager;
import org.eclipse.hawkbit.repository.jpa.model.JpaDistributionSetType;
import org.eclipse.hawkbit.repository.jpa.model.JpaSoftwareModuleType;
import org.eclipse.hawkbit.repository.model.DistributionSetType;
import org.eclipse.hawkbit.repository.model.SoftwareModuleType;
import org.eclipse.hawkbit.repository.model.TenantAwareBaseEntity;
import org.springframework.data.domain.Page;
import org.springframework.data.domain.Pageable;
@@ -40,6 +41,7 @@ public interface DistributionSetTypeRepository
* @param isDeleted
* to <code>true</code> if only soft deleted entries of
* <code>false</code> if undeleted ones
*
* @return list of found {@link DistributionSetType}s
*/
Page<JpaDistributionSetType> findByDeleted(Pageable pageable, boolean isDeleted);
@@ -47,7 +49,8 @@ public interface DistributionSetTypeRepository
/**
* @param isDeleted
* to <code>true</code> if only marked as deleted have to be
* count or all undeleted.
* counted or all undeleted.
*
* @return number of {@link DistributionSetType}s in the repository.
*/
long countByDeleted(boolean isDeleted);
@@ -59,6 +62,7 @@ public interface DistributionSetTypeRepository
* @param softwareModuleType
* the software module type to count the distribution set type
* which has this software module type assigned
*
* @return the number of {@link DistributionSetType}s in the repository
* assigned to the given software module type
*/
@@ -78,8 +82,29 @@ public interface DistributionSetTypeRepository
@Query("DELETE FROM JpaDistributionSetType t WHERE t.tenant = :tenant")
void deleteByTenant(@Param("tenant") String tenant);
/**
* Retrieves the {@link DistributionSetType}s for the given IDs. Workaround
* for https://bugs.eclipse.org/bugs/show_bug.cgi?id=349477
*
* @param ids
* of the types to be located
*
* @return a list of distribution set types
*/
@Override
// Workaround for https://bugs.eclipse.org/bugs/show_bug.cgi?id=349477
@Query("SELECT d FROM JpaDistributionSetType d WHERE d.id IN ?1")
List<JpaDistributionSetType> findAll(Iterable<Long> ids);
/**
* Counts the {@link SoftwareModuleType}s which are associated with the
* addressed {@link DistributionSetType}.
*
* @param id
* of the distribution set type
*
* @return the number of associated software module types
*/
@Query("SELECT COUNT (e.smType) FROM DistributionSetTypeElement e WHERE e.dsType.id = :id")
long countSmTypesById(@Param("id") Long id);
}

View File

@@ -17,15 +17,18 @@ import org.eclipse.hawkbit.artifact.repository.HashNotMatchException;
import org.eclipse.hawkbit.artifact.repository.model.AbstractDbArtifact;
import org.eclipse.hawkbit.artifact.repository.model.DbArtifactHash;
import org.eclipse.hawkbit.repository.ArtifactManagement;
import org.eclipse.hawkbit.repository.QuotaManagement;
import org.eclipse.hawkbit.repository.exception.ArtifactDeleteFailedException;
import org.eclipse.hawkbit.repository.exception.ArtifactUploadFailedException;
import org.eclipse.hawkbit.repository.exception.EntityAlreadyExistsException;
import org.eclipse.hawkbit.repository.exception.EntityNotFoundException;
import org.eclipse.hawkbit.repository.exception.InvalidMD5HashException;
import org.eclipse.hawkbit.repository.exception.InvalidSHA1HashException;
import org.eclipse.hawkbit.repository.exception.QuotaExceededException;
import org.eclipse.hawkbit.repository.jpa.configuration.Constants;
import org.eclipse.hawkbit.repository.jpa.model.JpaArtifact;
import org.eclipse.hawkbit.repository.jpa.model.JpaSoftwareModule;
import org.eclipse.hawkbit.repository.jpa.utils.QuotaHelper;
import org.eclipse.hawkbit.repository.model.Artifact;
import org.eclipse.hawkbit.repository.model.SoftwareModule;
import org.eclipse.hawkbit.tenancy.TenantAware;
@@ -49,6 +52,8 @@ public class JpaArtifactManagement implements ArtifactManagement {
private static final Logger LOG = LoggerFactory.getLogger(JpaArtifactManagement.class);
private static final String MAX_ARTIFACT_SIZE_EXCEEDED = "Quota exceeded: The artifact '%s' (%s bytes) which has been uploaded for software module '%s' exceeds the maximum artifact size of %s bytes.";
private final LocalArtifactRepository localArtifactRepository;
private final SoftwareModuleRepository softwareModuleRepository;
@@ -57,12 +62,15 @@ public class JpaArtifactManagement implements ArtifactManagement {
private final TenantAware tenantAware;
private final QuotaManagement quotaManagement;
JpaArtifactManagement(final LocalArtifactRepository localArtifactRepository,
final SoftwareModuleRepository softwareModuleRepository, final ArtifactRepository artifactRepository,
final TenantAware tenantAware) {
final QuotaManagement quotaManagement, final TenantAware tenantAware) {
this.localArtifactRepository = localArtifactRepository;
this.softwareModuleRepository = softwareModuleRepository;
this.artifactRepository = artifactRepository;
this.quotaManagement = quotaManagement;
this.tenantAware = tenantAware;
}
@@ -87,13 +95,16 @@ public class JpaArtifactManagement implements ArtifactManagement {
ConcurrencyFailureException.class }, maxAttempts = Constants.TX_RT_MAX, backoff = @Backoff(delay = Constants.TX_RT_DELAY))
public Artifact create(final InputStream stream, final long moduleId, final String filename,
final String providedMd5Sum, final String providedSha1Sum, final boolean overrideExisting,
final String contentType) {
final String contentType, final long filesize) {
AbstractDbArtifact result = null;
final SoftwareModule softwareModule = getModuleAndThrowExceptionIfThatFails(moduleId);
final Artifact existing = checkForExistingArtifact(filename, overrideExisting, softwareModule);
assertArtifactQuota(moduleId, 1);
assertMaxArtifactSizeQuota(filename, moduleId, filesize);
try {
result = artifactRepository.store(tenantAware.getCurrentTenant(), stream, filename, contentType,
new DbArtifactHash(providedSha1Sum, providedMd5Sum));
@@ -113,6 +124,23 @@ public class JpaArtifactManagement implements ArtifactManagement {
return storeArtifactMetadata(softwareModule, filename, result, existing);
}
private void assertArtifactQuota(final long id, final int requested) {
QuotaHelper.assertAssignmentQuota(id, requested, quotaManagement.getMaxArtifactsPerSoftwareModule(),
Artifact.class, SoftwareModule.class, localArtifactRepository::countBySoftwareModuleId);
}
private void assertMaxArtifactSizeQuota(final String filename, final long id, final long artifactSize) {
final long maxArtifactSize = quotaManagement.getMaxArtifactSize();
if (maxArtifactSize <= 0) {
return;
}
if (artifactSize > maxArtifactSize) {
final String msg = String.format(MAX_ARTIFACT_SIZE_EXCEEDED, filename, artifactSize, id, maxArtifactSize);
LOG.warn(msg);
throw new QuotaExceededException(msg);
}
}
@Override
@Transactional
@Retryable(include = {
@@ -207,8 +235,8 @@ public class JpaArtifactManagement implements ArtifactManagement {
@Retryable(include = {
ConcurrencyFailureException.class }, maxAttempts = Constants.TX_RT_MAX, backoff = @Backoff(delay = Constants.TX_RT_DELAY))
public Artifact create(final InputStream inputStream, final long moduleId, final String filename,
final boolean overrideExisting) {
return create(inputStream, moduleId, filename, null, null, overrideExisting, null);
final boolean overrideExisting, final long filesize) {
return create(inputStream, moduleId, filename, null, null, overrideExisting, null, filesize);
}
@Override

View File

@@ -44,7 +44,6 @@ import org.eclipse.hawkbit.repository.event.remote.TargetPollEvent;
import org.eclipse.hawkbit.repository.event.remote.entity.CancelTargetAssignmentEvent;
import org.eclipse.hawkbit.repository.exception.CancelActionNotAllowedException;
import org.eclipse.hawkbit.repository.exception.EntityNotFoundException;
import org.eclipse.hawkbit.repository.exception.QuotaExceededException;
import org.eclipse.hawkbit.repository.jpa.builder.JpaActionStatusCreate;
import org.eclipse.hawkbit.repository.jpa.configuration.Constants;
import org.eclipse.hawkbit.repository.jpa.executor.AfterTransactionCommitExecutor;
@@ -56,6 +55,7 @@ import org.eclipse.hawkbit.repository.jpa.model.JpaDistributionSet;
import org.eclipse.hawkbit.repository.jpa.model.JpaTarget;
import org.eclipse.hawkbit.repository.jpa.model.JpaTarget_;
import org.eclipse.hawkbit.repository.jpa.specifications.ActionSpecifications;
import org.eclipse.hawkbit.repository.jpa.utils.QuotaHelper;
import org.eclipse.hawkbit.repository.model.Action;
import org.eclipse.hawkbit.repository.model.Action.Status;
import org.eclipse.hawkbit.repository.model.ActionStatus;
@@ -535,8 +535,8 @@ public class JpaControllerManagement implements ControllerManagement {
break;
default:
// information status entry - check for a potential DOS attack
checkForTooManyStatusEntries(action);
checkForTooManyStatusMessages(actionStatus);
assertActionStatusQuota(action);
assertActionStatusMessageQuota(actionStatus);
break;
}
@@ -546,12 +546,9 @@ public class JpaControllerManagement implements ControllerManagement {
return action;
}
private void checkForTooManyStatusMessages(final JpaActionStatus actionStatus) {
if (actionStatus.getMessages().size() > quotaManagement.getMaxMessagesPerActionStatus()) {
throw new QuotaExceededException("ActionStatus messages", actionStatus.getMessages().size(),
quotaManagement.getMaxStatusEntriesPerAction());
}
private void assertActionStatusMessageQuota(final JpaActionStatus actionStatus) {
QuotaHelper.assertAssignmentQuota(actionStatus.getId(), actionStatus.getMessages().size(),
quotaManagement.getMaxMessagesPerActionStatus(), "Message", ActionStatus.class.getSimpleName(), null);
}
private void handleFinishedCancelation(final JpaActionStatus actionStatus, final JpaAction action) {
@@ -607,8 +604,8 @@ public class JpaControllerManagement implements ControllerManagement {
break;
default:
// information status entry - check for a potential DOS attack
checkForTooManyStatusEntries(action);
checkForTooManyStatusMessages(actionStatus);
assertActionStatusQuota(action);
assertActionStatusMessageQuota(actionStatus);
break;
}
@@ -628,16 +625,9 @@ public class JpaControllerManagement implements ControllerManagement {
targetRepository.save(mergedTarget);
}
private void checkForTooManyStatusEntries(final JpaAction action) {
if (quotaManagement.getMaxStatusEntriesPerAction() > 0) {
final Long statusCount = actionStatusRepository.countByAction(action);
if (statusCount >= quotaManagement.getMaxStatusEntriesPerAction()) {
throw new QuotaExceededException(ActionStatus.class, statusCount + 1,
quotaManagement.getMaxStatusEntriesPerAction());
}
}
private void assertActionStatusQuota(final JpaAction action) {
QuotaHelper.assertAssignmentQuota(action.getId(), 1, quotaManagement.getMaxStatusEntriesPerAction(),
ActionStatus.class, Action.class, actionStatusRepository::countByActionId);
}
private void handleFinishedAndStoreInTargetStatus(final JpaAction action) {
@@ -694,16 +684,17 @@ public class JpaControllerManagement implements ControllerManagement {
// unknown update mode
throw new IllegalStateException("The update mode " + updateMode + " is not supported.");
}
final int attributeCount = controllerAttributes.size();
if (attributeCount > quotaManagement.getMaxAttributeEntriesPerTarget()) {
throw new QuotaExceededException("Controller attributes", attributeCount,
quotaManagement.getMaxAttributeEntriesPerTarget());
}
assertTargetAttributesQuota(target);
return targetRepository.save(target);
}
private void assertTargetAttributesQuota(final JpaTarget target) {
final int limit = quotaManagement.getMaxAttributeEntriesPerTarget();
QuotaHelper.assertAssignmentQuota(target.getId(), target.getControllerAttributes().size(), limit, "Attribute",
Target.class.getSimpleName(), null);
}
@Override
@Transactional
@Retryable(include = {
@@ -775,8 +766,8 @@ public class JpaControllerManagement implements ControllerManagement {
final JpaActionStatus statusMessage = create.build();
statusMessage.setAction(action);
checkForTooManyStatusEntries(action);
checkForTooManyStatusMessages(statusMessage);
assertActionStatusQuota(action);
assertActionStatusMessageQuota(statusMessage);
return actionStatusRepository.save(statusMessage);
}

View File

@@ -28,6 +28,7 @@ import javax.persistence.criteria.Root;
import org.eclipse.hawkbit.repository.ActionFields;
import org.eclipse.hawkbit.repository.DeploymentManagement;
import org.eclipse.hawkbit.repository.QuotaManagement;
import org.eclipse.hawkbit.repository.RepositoryConstants;
import org.eclipse.hawkbit.repository.TargetManagement;
import org.eclipse.hawkbit.repository.TenantConfigurationManagement;
@@ -46,6 +47,7 @@ import org.eclipse.hawkbit.repository.jpa.model.JpaDistributionSet;
import org.eclipse.hawkbit.repository.jpa.model.JpaTarget;
import org.eclipse.hawkbit.repository.jpa.model.JpaTarget_;
import org.eclipse.hawkbit.repository.jpa.rsql.RSQLUtility;
import org.eclipse.hawkbit.repository.jpa.utils.QuotaHelper;
import org.eclipse.hawkbit.repository.model.Action;
import org.eclipse.hawkbit.repository.model.Action.ActionType;
import org.eclipse.hawkbit.repository.model.Action.Status;
@@ -115,6 +117,7 @@ public class JpaDeploymentManagement implements DeploymentManagement {
private final OnlineDsAssignmentStrategy onlineDsAssignmentStrategy;
private final OfflineDsAssignmentStrategy offlineDsAssignmentStrategy;
private final TenantConfigurationManagement tenantConfigurationManagement;
private final QuotaManagement quotaManagement;
private final SystemSecurityContext systemSecurityContext;
private final Database database;
@@ -124,7 +127,7 @@ public class JpaDeploymentManagement implements DeploymentManagement {
final AuditorAware<String> auditorProvider, final ApplicationEventPublisher eventPublisher,
final ApplicationContext applicationContext, final AfterTransactionCommitExecutor afterCommit,
final VirtualPropertyReplacer virtualPropertyReplacer, final PlatformTransactionManager txManager,
final TenantConfigurationManagement tenantConfigurationManagement,
final TenantConfigurationManagement tenantConfigurationManagement, final QuotaManagement quotaManagement,
final SystemSecurityContext systemSecurityContext, final Database database) {
this.entityManager = entityManager;
this.actionRepository = actionRepository;
@@ -139,10 +142,11 @@ public class JpaDeploymentManagement implements DeploymentManagement {
this.virtualPropertyReplacer = virtualPropertyReplacer;
this.txManager = txManager;
onlineDsAssignmentStrategy = new OnlineDsAssignmentStrategy(targetRepository, afterCommit, eventPublisher,
applicationContext, actionRepository, actionStatusRepository);
applicationContext, actionRepository, actionStatusRepository, quotaManagement);
offlineDsAssignmentStrategy = new OfflineDsAssignmentStrategy(targetRepository, afterCommit, eventPublisher,
applicationContext, actionRepository, actionStatusRepository);
applicationContext, actionRepository, actionStatusRepository, quotaManagement);
this.tenantConfigurationManagement = tenantConfigurationManagement;
this.quotaManagement = quotaManagement;
this.systemSecurityContext = systemSecurityContext;
this.database = database;
}
@@ -215,8 +219,8 @@ public class JpaDeploymentManagement implements DeploymentManagement {
* a list of all targets and their action type
* @param actionMessage
* an optional message to be written into the action status
* @param offline
* to <code>true</code> in offline case
* @param assignmentStrategy
* the assignment strategy (online /offline)
* @return the assignment result
*
* @throw IncompleteDistributionSetException if mandatory
@@ -240,6 +244,11 @@ public class JpaDeploymentManagement implements DeploymentManagement {
final List<String> controllerIDs = targetsWithActionType.stream().map(TargetWithActionType::getControllerId)
.collect(Collectors.toList());
// enforce the 'max targets per manual assignment' quota
if (!controllerIDs.isEmpty()) {
assertMaxTargetsPerManualAssignmentQuota(set.getId(), controllerIDs.size());
}
LOG.debug("assignDistribution({}) to {} targets", set, controllerIDs.size());
final Map<String, TargetWithActionType> targetsWithActionMap = targetsWithActionType.stream()
@@ -318,6 +327,20 @@ public class JpaDeploymentManagement implements DeploymentManagement {
targetManagement);
}
/**
* Enforces the quota defining the maximum number of {@link Target}s per
* manual {@link DistributionSet} assignment.
*
* @param id
* of the distribution set
* @param requested
* number of targets to check
*/
private void assertMaxTargetsPerManualAssignmentQuota(final Long id, final int requested) {
QuotaHelper.assertAssignmentQuota(id, requested, quotaManagement.getMaxTargetsPerManualAssignment(),
Target.class, DistributionSet.class, null);
}
@Override
@Transactional(isolation = Isolation.READ_COMMITTED)
@Retryable(include = {

View File

@@ -23,6 +23,7 @@ import org.eclipse.hawkbit.repository.DistributionSetManagement;
import org.eclipse.hawkbit.repository.DistributionSetMetadataFields;
import org.eclipse.hawkbit.repository.DistributionSetTagManagement;
import org.eclipse.hawkbit.repository.DistributionSetTypeManagement;
import org.eclipse.hawkbit.repository.QuotaManagement;
import org.eclipse.hawkbit.repository.SystemManagement;
import org.eclipse.hawkbit.repository.builder.DistributionSetCreate;
import org.eclipse.hawkbit.repository.builder.DistributionSetUpdate;
@@ -43,6 +44,7 @@ import org.eclipse.hawkbit.repository.jpa.model.JpaSoftwareModule;
import org.eclipse.hawkbit.repository.jpa.rsql.RSQLUtility;
import org.eclipse.hawkbit.repository.jpa.specifications.DistributionSetSpecification;
import org.eclipse.hawkbit.repository.jpa.specifications.SpecificationsBuilder;
import org.eclipse.hawkbit.repository.jpa.utils.QuotaHelper;
import org.eclipse.hawkbit.repository.model.Action;
import org.eclipse.hawkbit.repository.model.DistributionSet;
import org.eclipse.hawkbit.repository.model.DistributionSetFilter;
@@ -91,6 +93,8 @@ public class JpaDistributionSetManagement implements DistributionSetManagement {
private final DistributionSetTypeManagement distributionSetTypeManagement;
private final QuotaManagement quotaManagement;
private final DistributionSetMetadataRepository distributionSetMetadataRepository;
private final TargetFilterQueryRepository targetFilterQueryRepository;
@@ -118,7 +122,7 @@ public class JpaDistributionSetManagement implements DistributionSetManagement {
JpaDistributionSetManagement(final EntityManager entityManager,
final DistributionSetRepository distributionSetRepository,
final DistributionSetTagManagement distributionSetTagManagement, final SystemManagement systemManagement,
final DistributionSetTypeManagement distributionSetTypeManagement,
final DistributionSetTypeManagement distributionSetTypeManagement, final QuotaManagement quotaManagement,
final DistributionSetMetadataRepository distributionSetMetadataRepository,
final TargetFilterQueryRepository targetFilterQueryRepository, final ActionRepository actionRepository,
final NoCountPagingRepository criteriaNoCountDao, final ApplicationEventPublisher eventPublisher,
@@ -132,6 +136,7 @@ public class JpaDistributionSetManagement implements DistributionSetManagement {
this.distributionSetTagManagement = distributionSetTagManagement;
this.systemManagement = systemManagement;
this.distributionSetTypeManagement = distributionSetTypeManagement;
this.quotaManagement = quotaManagement;
this.distributionSetMetadataRepository = distributionSetMetadataRepository;
this.targetFilterQueryRepository = targetFilterQueryRepository;
this.actionRepository = actionRepository;
@@ -317,6 +322,9 @@ public class JpaDistributionSetManagement implements DistributionSetManagement {
assertDistributionSetIsNotAssignedToTargets(setId);
final JpaDistributionSet set = findDistributionSetAndThrowExceptionIfNotFound(setId);
assertSoftwareModuleQuota(setId, modules.size());
modules.forEach(set::addModule);
return distributionSetRepository.save(set);
@@ -452,6 +460,8 @@ public class JpaDistributionSetManagement implements DistributionSetManagement {
md.forEach(meta -> checkAndThrowAlreadyIfDistributionSetMetadataExists(
new DsMetadataCompositeKey(dsId, meta.getKey())));
assertMetaDataQuota(dsId, md.size());
final JpaDistributionSet set = touch(dsId);
return Collections.unmodifiableList(md.stream()
@@ -460,6 +470,18 @@ public class JpaDistributionSetManagement implements DistributionSetManagement {
.collect(Collectors.toList()));
}
private void assertMetaDataQuota(final Long dsId, final int requested) {
QuotaHelper.assertAssignmentQuota(dsId, requested, quotaManagement.getMaxMetaDataEntriesPerDistributionSet(),
DistributionSetMetadata.class, DistributionSet.class,
distributionSetMetadataRepository::countByDistributionSetId);
}
private void assertSoftwareModuleQuota(final Long id, final int requested) {
QuotaHelper.assertAssignmentQuota(id, requested, quotaManagement.getMaxSoftwareModulesPerDistributionSet(),
SoftwareModule.class, DistributionSet.class,
softwareModuleRepository::countByAssignedToId);
}
@Override
@Transactional
@Retryable(include = {
@@ -500,9 +522,8 @@ public class JpaDistributionSetManagement implements DistributionSetManagement {
private JpaDistributionSet touch(final DistributionSet ds) {
// merge base distribution set so optLockRevision gets updated and audit
// log written because
// modifying metadata is modifying the base distribution set itself for
// auditing purposes.
// log written because modifying metadata is modifying the base
// distribution set itself for auditing purposes.
final JpaDistributionSet result = entityManager.merge((JpaDistributionSet) ds);
result.setLastModifiedAt(0L);

View File

@@ -17,11 +17,13 @@ import java.util.stream.Collectors;
import org.eclipse.hawkbit.repository.DistributionSetTypeFields;
import org.eclipse.hawkbit.repository.DistributionSetTypeManagement;
import org.eclipse.hawkbit.repository.QuotaManagement;
import org.eclipse.hawkbit.repository.builder.DistributionSetTypeCreate;
import org.eclipse.hawkbit.repository.builder.DistributionSetTypeUpdate;
import org.eclipse.hawkbit.repository.builder.GenericDistributionSetTypeUpdate;
import org.eclipse.hawkbit.repository.exception.EntityNotFoundException;
import org.eclipse.hawkbit.repository.exception.EntityReadOnlyException;
import org.eclipse.hawkbit.repository.exception.QuotaExceededException;
import org.eclipse.hawkbit.repository.jpa.builder.JpaDistributionSetTypeCreate;
import org.eclipse.hawkbit.repository.jpa.configuration.Constants;
import org.eclipse.hawkbit.repository.jpa.model.JpaDistributionSetType;
@@ -29,6 +31,7 @@ import org.eclipse.hawkbit.repository.jpa.model.JpaSoftwareModuleType;
import org.eclipse.hawkbit.repository.jpa.rsql.RSQLUtility;
import org.eclipse.hawkbit.repository.jpa.specifications.DistributionSetTypeSpecification;
import org.eclipse.hawkbit.repository.jpa.specifications.SpecificationsBuilder;
import org.eclipse.hawkbit.repository.jpa.utils.QuotaHelper;
import org.eclipse.hawkbit.repository.model.DistributionSetType;
import org.eclipse.hawkbit.repository.model.SoftwareModuleType;
import org.eclipse.hawkbit.repository.rsql.VirtualPropertyReplacer;
@@ -64,17 +67,20 @@ public class JpaDistributionSetTypeManagement implements DistributionSetTypeMana
private final NoCountPagingRepository criteriaNoCountDao;
private final Database database;
private final QuotaManagement quotaManagement;
JpaDistributionSetTypeManagement(final DistributionSetTypeRepository distributionSetTypeRepository,
final SoftwareModuleTypeRepository softwareModuleTypeRepository,
final DistributionSetRepository distributionSetRepository,
final VirtualPropertyReplacer virtualPropertyReplacer, final NoCountPagingRepository criteriaNoCountDao,
final Database database) {
final Database database, final QuotaManagement quotaManagement) {
this.distributionSetTypeRepository = distributionSetTypeRepository;
this.softwareModuleTypeRepository = softwareModuleTypeRepository;
this.distributionSetRepository = distributionSetRepository;
this.virtualPropertyReplacer = virtualPropertyReplacer;
this.criteriaNoCountDao = criteriaNoCountDao;
this.database = database;
this.quotaManagement = quotaManagement;
}
@Override
@@ -116,6 +122,7 @@ public class JpaDistributionSetTypeManagement implements DistributionSetTypeMana
final JpaDistributionSetType type = findDistributionSetTypeAndThrowExceptionIfNotFound(dsTypeId);
checkDistributionSetTypeSoftwareModuleTypesIsAllowedToModify(dsTypeId);
assertSoftwareModuleTypeQuota(dsTypeId, softwareModulesTypeIds.size());
modules.forEach(type::addMandatoryModuleType);
@@ -138,11 +145,31 @@ public class JpaDistributionSetTypeManagement implements DistributionSetTypeMana
final JpaDistributionSetType type = findDistributionSetTypeAndThrowExceptionIfNotFound(dsTypeId);
checkDistributionSetTypeSoftwareModuleTypesIsAllowedToModify(dsTypeId);
assertSoftwareModuleTypeQuota(dsTypeId, softwareModulesTypeIds.size());
modules.forEach(type::addOptionalModuleType);
return distributionSetTypeRepository.save(type);
}
/**
* Enforces the quota specifiying the maximum number of
* {@link SoftwareModuleType}s per {@link DistributionSetType}.
*
* @param id
* of the distribution set type
* @param requested
* number of software module types to check
*
* @throws QuotaExceededException
* if the software module type quota is exceeded
*/
private void assertSoftwareModuleTypeQuota(final long id, final int requested) {
QuotaHelper.assertAssignmentQuota(id, requested,
quotaManagement.getMaxSoftwareModuleTypesPerDistributionSetType(), SoftwareModuleType.class,
DistributionSetType.class, distributionSetTypeRepository::countSmTypesById);
}
@Override
@Transactional
@Retryable(include = {

View File

@@ -52,6 +52,7 @@ import org.eclipse.hawkbit.repository.jpa.rollout.condition.RolloutGroupConditio
import org.eclipse.hawkbit.repository.jpa.rsql.RSQLUtility;
import org.eclipse.hawkbit.repository.jpa.specifications.RolloutSpecification;
import org.eclipse.hawkbit.repository.jpa.specifications.SpecificationsBuilder;
import org.eclipse.hawkbit.repository.jpa.utils.QuotaHelper;
import org.eclipse.hawkbit.repository.model.Action;
import org.eclipse.hawkbit.repository.model.Action.ActionType;
import org.eclipse.hawkbit.repository.model.Action.Status;
@@ -226,11 +227,15 @@ public class JpaRolloutManagement extends AbstractRolloutManagement {
}
private Rollout createRolloutGroups(final int amountOfGroups, final RolloutGroupConditions conditions,
final Rollout rollout) {
final JpaRollout rollout) {
RolloutHelper.verifyRolloutInStatus(rollout, RolloutStatus.CREATING);
RolloutHelper.verifyRolloutGroupConditions(conditions);
final JpaRollout savedRollout = (JpaRollout) rollout;
final JpaRollout savedRollout = rollout;
// we can enforce the 'max targets per group' quota right here because
// we want to distribute the targets equally to the different groups
assertTargetsPerRolloutGroupQuota(rollout.getTotalTargets() / amountOfGroups);
RolloutGroup lastSavedGroup = null;
for (int i = 0; i < amountOfGroups; i++) {
@@ -269,7 +274,7 @@ public class JpaRolloutManagement extends AbstractRolloutManagement {
RolloutHelper.verifyRolloutInStatus(rollout, RolloutStatus.CREATING);
final JpaRollout savedRollout = (JpaRollout) rollout;
// Preparing the groups
// prepare the groups
final List<RolloutGroup> groups = groupList.stream()
.map(group -> JpaRolloutHelper.prepareRolloutGroupWithDefaultConditions(group, conditions))
.collect(Collectors.toList());
@@ -278,7 +283,13 @@ public class JpaRolloutManagement extends AbstractRolloutManagement {
RolloutHelper.verifyRemainingTargets(
calculateRemainingTargets(groups, savedRollout.getTargetFilterQuery(), savedRollout.getCreatedAt()));
// Persisting the groups
// check if we need to enforce the 'max targets per group' quota
if (quotaManagement.getMaxTargetsPerRolloutGroup() > 0) {
validateTargetsInGroups(groups, savedRollout.getTargetFilterQuery(), savedRollout.getCreatedAt())
.getTargetsPerGroup().forEach(this::assertTargetsPerRolloutGroupQuota);
}
// create and persist the groups (w/o filling them with targets)
RolloutGroup lastSavedGroup = null;
for (final RolloutGroup srcGroup : groups) {
final JpaRolloutGroup group = new JpaRolloutGroup();
@@ -346,7 +357,7 @@ public class JpaRolloutManagement extends AbstractRolloutManagement {
// When all groups are ready the rollout status can be changed to be
// ready, too.
if (readyGroups == rolloutGroups.size()) {
LOGGER.debug("rollout {} creatin done. Switch to READY.", rollout.getId());
LOGGER.debug("rollout {} creation done. Switch to READY.", rollout.getId());
rollout.setStatus(RolloutStatus.READY);
rollout.setLastCheck(0);
rollout.setTotalTargets(totalTargets);
@@ -383,9 +394,10 @@ public class JpaRolloutManagement extends AbstractRolloutManagement {
return rolloutGroupRepository.save(group);
}
long targetsLeftToAdd = expectedInGroup - currentlyInGroup;
try {
long targetsLeftToAdd = expectedInGroup - currentlyInGroup;
do {
// Add up to TRANSACTION_TARGETS of the left targets
// In case a TransactionException is thrown this loop aborts
@@ -558,6 +570,9 @@ public class JpaRolloutManagement extends AbstractRolloutManagement {
final List<Long> targetIds = targets.stream().map(Target::getId).collect(Collectors.toList());
actionRepository.switchStatus(Action.Status.CANCELED, targetIds, false, Action.Status.SCHEDULED);
targets.forEach(target -> {
assertActionsPerTargetQuota(target, 1);
final JpaAction action = new JpaAction();
action.setTarget(target);
action.setActive(false);
@@ -1006,6 +1021,11 @@ public class JpaRolloutManagement extends AbstractRolloutManagement {
return rollout;
}
@Override
public boolean exists(final long rolloutId) {
return rolloutRepository.exists(rolloutId);
}
private Map<Long, List<TotalTargetCountActionStatus>> getStatusCountItemForRollout(final List<Long> rollouts) {
if (rollouts.isEmpty()) {
return null;
@@ -1044,8 +1064,33 @@ public class JpaRolloutManagement extends AbstractRolloutManagement {
}
}
@Override
public boolean exists(final long rolloutId) {
return rolloutRepository.exists(rolloutId);
/**
* Enforces the quota defining the maximum number of {@link Target}s per
* {@link RolloutGroup}.
*
* @param group
* The rollout group
* @param requested
* number of targets to check
*/
private void assertTargetsPerRolloutGroupQuota(final long requested) {
final int quota = quotaManagement.getMaxTargetsPerRolloutGroup();
QuotaHelper.assertAssignmentQuota(requested, quota, Target.class, RolloutGroup.class);
}
/**
* Enforces the quota defining the maximum number of {@link Action}s per
* {@link Target}.
*
* @param target
* The target
* @param requested
* number of actions to check
*/
private void assertActionsPerTargetQuota(final Target target, final int requested) {
final int quota = quotaManagement.getMaxActionsPerTarget();
QuotaHelper.assertAssignmentQuota(target.getId(), requested, quota, Action.class, Target.class,
actionRepository::countByTargetId);
}
}

View File

@@ -14,6 +14,7 @@ import java.util.Collection;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import java.util.stream.Collectors;
@@ -28,6 +29,7 @@ import javax.persistence.criteria.Root;
import org.eclipse.hawkbit.im.authentication.SpPermission.SpringEvalExpressions;
import org.eclipse.hawkbit.repository.ArtifactManagement;
import org.eclipse.hawkbit.repository.QuotaManagement;
import org.eclipse.hawkbit.repository.RepositoryConstants;
import org.eclipse.hawkbit.repository.SoftwareModuleFields;
import org.eclipse.hawkbit.repository.SoftwareModuleManagement;
@@ -53,6 +55,7 @@ import org.eclipse.hawkbit.repository.jpa.model.SwMetadataCompositeKey;
import org.eclipse.hawkbit.repository.jpa.rsql.RSQLUtility;
import org.eclipse.hawkbit.repository.jpa.specifications.SoftwareModuleSpecification;
import org.eclipse.hawkbit.repository.jpa.specifications.SpecificationsBuilder;
import org.eclipse.hawkbit.repository.jpa.utils.QuotaHelper;
import org.eclipse.hawkbit.repository.model.Artifact;
import org.eclipse.hawkbit.repository.model.AssignedSoftwareModule;
import org.eclipse.hawkbit.repository.model.DistributionSet;
@@ -103,7 +106,10 @@ public class JpaSoftwareModuleManagement implements SoftwareModuleManagement {
private final ArtifactManagement artifactManagement;
private final QuotaManagement quotaManagement;
private final VirtualPropertyReplacer virtualPropertyReplacer;
private final Database database;
JpaSoftwareModuleManagement(final EntityManager entityManager,
@@ -112,8 +118,8 @@ public class JpaSoftwareModuleManagement implements SoftwareModuleManagement {
final SoftwareModuleMetadataRepository softwareModuleMetadataRepository,
final SoftwareModuleTypeRepository softwareModuleTypeRepository,
final NoCountPagingRepository criteriaNoCountDao, final AuditorAware<String> auditorProvider,
final ArtifactManagement artifactManagement, final VirtualPropertyReplacer virtualPropertyReplacer,
final Database database) {
final ArtifactManagement artifactManagement, final QuotaManagement quotaManagement,
final VirtualPropertyReplacer virtualPropertyReplacer, final Database database) {
this.entityManager = entityManager;
this.distributionSetRepository = distributionSetRepository;
this.softwareModuleRepository = softwareModuleRepository;
@@ -122,6 +128,7 @@ public class JpaSoftwareModuleManagement implements SoftwareModuleManagement {
this.criteriaNoCountDao = criteriaNoCountDao;
this.auditorProvider = auditorProvider;
this.artifactManagement = artifactManagement;
this.quotaManagement = quotaManagement;
this.virtualPropertyReplacer = virtualPropertyReplacer;
this.database = database;
}
@@ -461,19 +468,13 @@ public class JpaSoftwareModuleManagement implements SoftwareModuleManagement {
@Retryable(include = {
ConcurrencyFailureException.class }, maxAttempts = Constants.TX_RT_MAX, backoff = @Backoff(delay = Constants.TX_RT_DELAY))
public SoftwareModuleMetadata createMetaData(final SoftwareModuleMetadataCreate c) {
final JpaSoftwareModuleMetadataCreate create = (JpaSoftwareModuleMetadataCreate) c;
final Long moduleId = create.getSoftwareModuleId();
assertSoftwareModuleExists(moduleId);
assertMetaDataQuota(moduleId, 1);
checkAndThrowAlreadyIfSoftwareModuleMetadataExists(create.getSoftwareModuleId(), create);
touch(create.getSoftwareModuleId());
return softwareModuleMetadataRepository.save(create.build());
}
private void checkAndThrowAlreadyIfSoftwareModuleMetadataExists(final Long moduleId,
final JpaSoftwareModuleMetadataCreate md) {
if (softwareModuleMetadataRepository.exists(new SwMetadataCompositeKey(moduleId, md.getKey()))) {
throwMetadataKeyAlreadyExists(md.getKey());
}
return saveMetadata(create);
}
@Override
@@ -482,7 +483,71 @@ public class JpaSoftwareModuleManagement implements SoftwareModuleManagement {
ConcurrencyFailureException.class }, maxAttempts = Constants.TX_RT_MAX, backoff = @Backoff(delay = Constants.TX_RT_DELAY))
public List<SoftwareModuleMetadata> createMetaData(final Collection<SoftwareModuleMetadataCreate> create) {
return create.stream().map(this::createMetaData).collect(Collectors.toList());
if (!create.isEmpty()) {
// check if all meta data entries refer to the same software module
final Long moduleId = ((JpaSoftwareModuleMetadataCreate) create.iterator().next()).getSoftwareModuleId();
if (createJpaMetadataCreateStream(create).allMatch(c -> moduleId.equals(c.getSoftwareModuleId()))) {
assertSoftwareModuleExists(moduleId);
assertMetaDataQuota(moduleId, create.size());
return createJpaMetadataCreateStream(create).map(this::saveMetadata).collect(Collectors.toList());
} else {
// group by software module id to minimize database access
final Map<Long, List<JpaSoftwareModuleMetadataCreate>> groups = createJpaMetadataCreateStream(create)
.collect(Collectors.groupingBy(JpaSoftwareModuleMetadataCreate::getSoftwareModuleId));
return groups.entrySet().stream().flatMap(e -> {
final Long id = e.getKey();
final List<JpaSoftwareModuleMetadataCreate> group = e.getValue();
assertSoftwareModuleExists(id);
assertMetaDataQuota(id, group.size());
return group.stream().map(this::saveMetadata);
}).collect(Collectors.toList());
}
}
return Collections.emptyList();
}
private static Stream<JpaSoftwareModuleMetadataCreate> createJpaMetadataCreateStream(
final Collection<SoftwareModuleMetadataCreate> create) {
return create.stream().map(c -> (JpaSoftwareModuleMetadataCreate) c);
}
private SoftwareModuleMetadata saveMetadata(final JpaSoftwareModuleMetadataCreate create) {
assertSoftwareModuleMetadataDoesNotExist(create.getSoftwareModuleId(), create);
return softwareModuleMetadataRepository.save(create.build());
}
private void assertSoftwareModuleMetadataDoesNotExist(final Long moduleId,
final JpaSoftwareModuleMetadataCreate md) {
if (softwareModuleMetadataRepository.exists(new SwMetadataCompositeKey(moduleId, md.getKey()))) {
throwMetadataKeyAlreadyExists(md.getKey());
}
}
private void assertSoftwareModuleExists(final Long moduleId) {
touch(moduleId);
}
/**
* Asserts the meta data quota for the software module with the given ID.
*
* @param moduleId
* The software module ID.
* @param requested
* Number of meta data entries to be created.
*/
private void assertMetaDataQuota(final Long moduleId, final int requested) {
final int maxMetaData = quotaManagement.getMaxMetaDataEntriesPerSoftwareModule();
QuotaHelper.assertAssignmentQuota(moduleId, requested, maxMetaData, SoftwareModuleMetadata.class,
SoftwareModule.class, softwareModuleMetadataRepository::countBySoftwareModuleId);
}
@Override
@@ -514,9 +579,8 @@ public class JpaSoftwareModuleManagement implements SoftwareModuleManagement {
*/
private JpaSoftwareModule touch(final SoftwareModule latestModule) {
// merge base distribution set so optLockRevision gets updated and audit
// log written because
// modifying metadata is modifying the base distribution set itself for
// auditing purposes.
// log written because modifying metadata is modifying the base
// distribution set itself for auditing purposes.
final JpaSoftwareModule result = entityManager.merge((JpaSoftwareModule) latestModule);
result.setLastModifiedAt(0L);

View File

@@ -14,6 +14,7 @@ import java.util.List;
import java.util.Optional;
import org.eclipse.hawkbit.repository.DistributionSetManagement;
import org.eclipse.hawkbit.repository.QuotaManagement;
import org.eclipse.hawkbit.repository.TargetFields;
import org.eclipse.hawkbit.repository.TargetFilterQueryFields;
import org.eclipse.hawkbit.repository.TargetFilterQueryManagement;
@@ -28,7 +29,9 @@ import org.eclipse.hawkbit.repository.jpa.model.JpaTargetFilterQuery;
import org.eclipse.hawkbit.repository.jpa.rsql.RSQLUtility;
import org.eclipse.hawkbit.repository.jpa.specifications.SpecificationsBuilder;
import org.eclipse.hawkbit.repository.jpa.specifications.TargetFilterQuerySpecification;
import org.eclipse.hawkbit.repository.jpa.utils.QuotaHelper;
import org.eclipse.hawkbit.repository.model.DistributionSet;
import org.eclipse.hawkbit.repository.model.Target;
import org.eclipse.hawkbit.repository.model.TargetFilterQuery;
import org.eclipse.hawkbit.repository.rsql.VirtualPropertyReplacer;
import org.springframework.dao.ConcurrencyFailureException;
@@ -56,18 +59,24 @@ import com.google.common.collect.Lists;
public class JpaTargetFilterQueryManagement implements TargetFilterQueryManagement {
private final TargetFilterQueryRepository targetFilterQueryRepository;
private final TargetRepository targetRepository;
private final VirtualPropertyReplacer virtualPropertyReplacer;
private final DistributionSetManagement distributionSetManagement;
private final QuotaManagement quotaManagement;
private final Database database;
JpaTargetFilterQueryManagement(final TargetFilterQueryRepository targetFilterQueryRepository,
final VirtualPropertyReplacer virtualPropertyReplacer,
final DistributionSetManagement distributionSetManagement, final Database database) {
final TargetRepository targetRepository, final VirtualPropertyReplacer virtualPropertyReplacer,
final DistributionSetManagement distributionSetManagement, final QuotaManagement quotaManagement,
final Database database) {
this.targetFilterQueryRepository = targetFilterQueryRepository;
this.targetRepository = targetRepository;
this.virtualPropertyReplacer = virtualPropertyReplacer;
this.distributionSetManagement = distributionSetManagement;
this.quotaManagement = quotaManagement;
this.database = database;
}
@@ -78,6 +87,10 @@ public class JpaTargetFilterQueryManagement implements TargetFilterQueryManageme
public TargetFilterQuery create(final TargetFilterQueryCreate c) {
final JpaTargetFilterQueryCreate create = (JpaTargetFilterQueryCreate) c;
// enforce the 'max targets per auto assign' quota right here even if
// the result of the filter query can vary over time
create.getSet().flatMap(set -> create.getQuery()).ifPresent(this::assertMaxTargetsQuota);
return targetFilterQueryRepository.save(create.build());
}
@@ -187,7 +200,18 @@ public class JpaTargetFilterQueryManagement implements TargetFilterQueryManageme
final JpaTargetFilterQuery targetFilterQuery = findTargetFilterQueryOrThrowExceptionIfNotFound(update.getId());
update.getName().ifPresent(targetFilterQuery::setName);
update.getQuery().ifPresent(targetFilterQuery::setQuery);
update.getQuery().ifPresent(query -> {
// enforce the 'max targets per auto assignment'-quota only if the
// query is going to change
if (targetFilterQuery.getAutoAssignDistributionSet() != null
&& !query.equals(targetFilterQuery.getQuery())) {
assertMaxTargetsQuota(query);
}
// set the new query
targetFilterQuery.setQuery(query);
});
return targetFilterQueryRepository.save(targetFilterQuery);
}
@@ -200,6 +224,13 @@ public class JpaTargetFilterQueryManagement implements TargetFilterQueryManageme
targetFilterQuery.setAutoAssignDistributionSet(
Optional.ofNullable(dsId).map(this::findDistributionSetAndThrowExceptionIfNotFound).orElse(null));
// we cannot be sure that the quota was enforced at creation time
// because the Target Filter Query REST API does not allow to specify an
// auto-assign distribution set when creating a target filter query
if (dsId != null) {
assertMaxTargetsQuota(targetFilterQuery.getQuery());
}
return targetFilterQueryRepository.save(targetFilterQuery);
}
@@ -219,4 +250,10 @@ public class JpaTargetFilterQueryManagement implements TargetFilterQueryManageme
return true;
}
private void assertMaxTargetsQuota(final String query) {
QuotaHelper.assertAssignmentQuota(
targetRepository.count(RSQLUtility.parse(query, TargetFields.class, virtualPropertyReplacer, database)),
quotaManagement.getMaxTargetsPerAutoAssignment(), Target.class, TargetFilterQuery.class);
}
}

View File

@@ -93,12 +93,23 @@ public interface LocalArtifactRepository extends BaseEntityRepository<JpaArtifac
*
* @param pageReq
* Pageable
* @param swId
* @param softwareModuleId
* software module id
*
* @return Page<Artifact>
*/
Page<Artifact> findBySoftwareModuleId(Pageable pageReq, final Long swId);
Page<Artifact> findBySoftwareModuleId(Pageable pageReq, Long softwareModuleId);
/**
* Count the artifacts that are associated with the given software module.
*
* @param softwareModuleId
* software module ID
*
* @return the current number of artifacts associated with the software
* module.
*/
long countBySoftwareModuleId(Long softwareModuleId);
/**
* Searches for a {@link Artifact} based user provided filename at upload
@@ -108,8 +119,9 @@ public interface LocalArtifactRepository extends BaseEntityRepository<JpaArtifac
* to search
* @param softwareModuleId
* selected software module id
*
* @return list of {@link Artifact}.
*/
Optional<Artifact> findFirstByFilenameAndSoftwareModuleId(final String filename, final Long softwareModuleId);
Optional<Artifact> findFirstByFilenameAndSoftwareModuleId(String filename, Long softwareModuleId);
}

View File

@@ -15,6 +15,7 @@ import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import org.eclipse.hawkbit.repository.QuotaManagement;
import org.eclipse.hawkbit.repository.RepositoryConstants;
import org.eclipse.hawkbit.repository.jpa.configuration.Constants;
import org.eclipse.hawkbit.repository.jpa.executor.AfterTransactionCommitExecutor;
@@ -43,9 +44,9 @@ public class OfflineDsAssignmentStrategy extends AbstractDsAssignmentStrategy {
OfflineDsAssignmentStrategy(final TargetRepository targetRepository,
final AfterTransactionCommitExecutor afterCommit, final ApplicationEventPublisher eventPublisher,
final ApplicationContext applicationContext, final ActionRepository actionRepository,
final ActionStatusRepository actionStatusRepository) {
final ActionStatusRepository actionStatusRepository, final QuotaManagement quotaManagement) {
super(targetRepository, afterCommit, eventPublisher, applicationContext, actionRepository,
actionStatusRepository);
actionStatusRepository, quotaManagement);
}
@Override

View File

@@ -14,6 +14,7 @@ import java.util.Map;
import java.util.Set;
import java.util.stream.Collectors;
import org.eclipse.hawkbit.repository.QuotaManagement;
import org.eclipse.hawkbit.repository.jpa.configuration.Constants;
import org.eclipse.hawkbit.repository.jpa.executor.AfterTransactionCommitExecutor;
import org.eclipse.hawkbit.repository.jpa.model.JpaAction;
@@ -41,9 +42,9 @@ public class OnlineDsAssignmentStrategy extends AbstractDsAssignmentStrategy {
OnlineDsAssignmentStrategy(final TargetRepository targetRepository,
final AfterTransactionCommitExecutor afterCommit, final ApplicationEventPublisher eventPublisher,
final ApplicationContext applicationContext, final ActionRepository actionRepository,
final ActionStatusRepository actionStatusRepository) {
final ActionStatusRepository actionStatusRepository, final QuotaManagement quotaManagement) {
super(targetRepository, afterCommit, eventPublisher, applicationContext, actionRepository,
actionStatusRepository);
actionStatusRepository, quotaManagement);
}
@Override

View File

@@ -23,6 +23,7 @@ import org.eclipse.hawkbit.repository.DistributionSetTagManagement;
import org.eclipse.hawkbit.repository.DistributionSetTypeManagement;
import org.eclipse.hawkbit.repository.EntityFactory;
import org.eclipse.hawkbit.repository.PropertiesQuotaManagement;
import org.eclipse.hawkbit.repository.QuotaManagement;
import org.eclipse.hawkbit.repository.RepositoryDefaultConfiguration;
import org.eclipse.hawkbit.repository.RepositoryProperties;
import org.eclipse.hawkbit.repository.RolloutGroupManagement;
@@ -69,6 +70,7 @@ import org.eclipse.hawkbit.repository.model.DistributionSet;
import org.eclipse.hawkbit.repository.model.DistributionSetType;
import org.eclipse.hawkbit.repository.model.Rollout;
import org.eclipse.hawkbit.repository.model.SoftwareModule;
import org.eclipse.hawkbit.repository.model.Target;
import org.eclipse.hawkbit.repository.model.TargetFilterQuery;
import org.eclipse.hawkbit.repository.model.helper.SystemManagementHolder;
import org.eclipse.hawkbit.repository.model.helper.TenantConfigurationManagementHolder;
@@ -376,7 +378,7 @@ public class RepositoryApplicationConfiguration extends JpaBaseConfiguration {
DistributionSetManagement distributionSetManagement(final EntityManager entityManager,
final DistributionSetRepository distributionSetRepository,
final DistributionSetTagManagement distributionSetTagManagement, final SystemManagement systemManagement,
final DistributionSetTypeManagement distributionSetTypeManagement,
final DistributionSetTypeManagement distributionSetTypeManagement, final QuotaManagement quotaManagement,
final DistributionSetMetadataRepository distributionSetMetadataRepository,
final TargetFilterQueryRepository targetFilterQueryRepository, final ActionRepository actionRepository,
final NoCountPagingRepository criteriaNoCountDao, final ApplicationEventPublisher eventPublisher,
@@ -386,7 +388,7 @@ public class RepositoryApplicationConfiguration extends JpaBaseConfiguration {
final DistributionSetTagRepository distributionSetTagRepository,
final AfterTransactionCommitExecutor afterCommit, final JpaProperties properties) {
return new JpaDistributionSetManagement(entityManager, distributionSetRepository, distributionSetTagManagement,
systemManagement, distributionSetTypeManagement, distributionSetMetadataRepository,
systemManagement, distributionSetTypeManagement, quotaManagement, distributionSetMetadataRepository,
targetFilterQueryRepository, actionRepository, criteriaNoCountDao, eventPublisher, applicationContext,
tenantAware, virtualPropertyReplacer, softwareModuleRepository, distributionSetTagRepository,
afterCommit, properties.getDatabase());
@@ -405,9 +407,10 @@ public class RepositoryApplicationConfiguration extends JpaBaseConfiguration {
final SoftwareModuleTypeRepository softwareModuleTypeRepository,
final DistributionSetRepository distributionSetRepository,
final VirtualPropertyReplacer virtualPropertyReplacer, final NoCountPagingRepository criteriaNoCountDao,
final JpaProperties properties) {
final JpaProperties properties, final QuotaManagement quotaManagement) {
return new JpaDistributionSetTypeManagement(distributionSetTypeRepository, softwareModuleTypeRepository,
distributionSetRepository, virtualPropertyReplacer, criteriaNoCountDao, properties.getDatabase());
distributionSetRepository, virtualPropertyReplacer, criteriaNoCountDao, properties.getDatabase(),
quotaManagement);
}
/**
@@ -457,22 +460,29 @@ public class RepositoryApplicationConfiguration extends JpaBaseConfiguration {
* {@link JpaTargetFilterQueryManagement} bean.
*
* @param targetFilterQueryRepository
* to query entity access
* holding {@link TargetFilterQuery} entities
* @param targetRepository
* holding {@link Target} entities
* @param virtualPropertyReplacer
* for RSQL handling
* @param distributionSetManagement
* for auto assign DS access
*
* @param quotaManagement
* to access quotas
* @param properties
* JPA properties
*
* @return a new {@link TargetFilterQueryManagement}
*/
@Bean
@ConditionalOnMissingBean
TargetFilterQueryManagement targetFilterQueryManagement(
final TargetFilterQueryRepository targetFilterQueryRepository,
final TargetFilterQueryRepository targetFilterQueryRepository, final TargetRepository targetRepository,
final VirtualPropertyReplacer virtualPropertyReplacer,
final DistributionSetManagement distributionSetManagement, final JpaProperties properties) {
return new JpaTargetFilterQueryManagement(targetFilterQueryRepository, virtualPropertyReplacer,
distributionSetManagement, properties.getDatabase());
final DistributionSetManagement distributionSetManagement, final QuotaManagement quotaManagement,
final JpaProperties properties) {
return new JpaTargetFilterQueryManagement(targetFilterQueryRepository, targetRepository,
virtualPropertyReplacer, distributionSetManagement, quotaManagement, properties.getDatabase());
}
/**
@@ -518,11 +528,11 @@ public class RepositoryApplicationConfiguration extends JpaBaseConfiguration {
final SoftwareModuleMetadataRepository softwareModuleMetadataRepository,
final SoftwareModuleTypeRepository softwareModuleTypeRepository,
final NoCountPagingRepository criteriaNoCountDao, final AuditorAware<String> auditorProvider,
final ArtifactManagement artifactManagement, final VirtualPropertyReplacer virtualPropertyReplacer,
final JpaProperties properties) {
final ArtifactManagement artifactManagement, final QuotaManagement quotaManagement,
final VirtualPropertyReplacer virtualPropertyReplacer, final JpaProperties properties) {
return new JpaSoftwareModuleManagement(entityManager, distributionSetRepository, softwareModuleRepository,
softwareModuleMetadataRepository, softwareModuleTypeRepository, criteriaNoCountDao, auditorProvider,
artifactManagement, virtualPropertyReplacer, properties.getDatabase());
artifactManagement, quotaManagement, virtualPropertyReplacer, properties.getDatabase());
}
/**
@@ -585,12 +595,12 @@ public class RepositoryApplicationConfiguration extends JpaBaseConfiguration {
final ApplicationEventPublisher eventPublisher, final ApplicationContext applicationContext,
final AfterTransactionCommitExecutor afterCommit, final VirtualPropertyReplacer virtualPropertyReplacer,
final PlatformTransactionManager txManager,
final TenantConfigurationManagement tenantConfigurationManagement,
final TenantConfigurationManagement tenantConfigurationManagement, final QuotaManagement quotaManagement,
final SystemSecurityContext systemSecurityContext, final JpaProperties properties) {
return new JpaDeploymentManagement(entityManager, actionRepository, distributionSetRepository, targetRepository,
actionStatusRepository, targetManagement, auditorProvider, eventPublisher, applicationContext,
afterCommit, virtualPropertyReplacer, txManager, tenantConfigurationManagement, systemSecurityContext,
properties.getDatabase());
afterCommit, virtualPropertyReplacer, txManager, tenantConfigurationManagement, quotaManagement,
systemSecurityContext, properties.getDatabase());
}
/**
@@ -609,9 +619,9 @@ public class RepositoryApplicationConfiguration extends JpaBaseConfiguration {
@ConditionalOnMissingBean
ArtifactManagement artifactManagement(final LocalArtifactRepository localArtifactRepository,
final SoftwareModuleRepository softwareModuleRepository, final ArtifactRepository artifactRepository,
final TenantAware tenantAware) {
final QuotaManagement quotaManagement, final TenantAware tenantAware) {
return new JpaArtifactManagement(localArtifactRepository, softwareModuleRepository, artifactRepository,
tenantAware);
quotaManagement, tenantAware);
}
/**

View File

@@ -30,11 +30,49 @@ public interface SoftwareModuleMetadataRepository
extends PagingAndSortingRepository<JpaSoftwareModuleMetadata, SwMetadataCompositeKey>,
JpaSpecificationExecutor<JpaSoftwareModuleMetadata> {
/**
* Locates the meta data entries that match the given software module ID and
* target visibility flag.
*
* @param page
* The pagination parameters.
* @param moduleId
* The ID of the software module.
* @param targetVisible
* The target visibility flag.
*
* @return A {@link Page} with the matching meta data entries.
*/
Page<JpaSoftwareModuleMetadata> findBySoftwareModuleIdAndTargetVisible(Pageable page, Long moduleId,
boolean targetVisible);
/**
* Locates the meta data entries that match the given software module IDs
* and target visibility flag.
*
* @param page
* The pagination parameters.
* @param moduleId
* List of software module IDs.
* @param targetVisible
* The target visibility flag.
*
* @return A {@link Page} with the matching meta data entries.
*/
@Query("SELECT smd.softwareModule.id, smd FROM JpaSoftwareModuleMetadata smd WHERE smd.softwareModule.id IN :moduleId AND smd.targetVisible = :targetVisible")
Page<Object[]> findBySoftwareModuleIdInAndTargetVisible(Pageable page, @Param("moduleId") Collection<Long> moduleId,
@Param("targetVisible") boolean targetVisible);
/**
* Counts the meta data entries that are associated with the addressed
* software module.
*
* @param moduleId
* The ID of the software module.
*
* @return The number of meta data entries associated with the software
* module.
*/
long countBySoftwareModuleId(@Param("moduleId") Long moduleId);
}

View File

@@ -87,6 +87,18 @@ public interface SoftwareModuleRepository
*/
Page<SoftwareModule> findByAssignedToId(Pageable pageable, Long setId);
/**
* Count the software modules which are assigned to the distribution set
* with the given ID.
*
* @param setId
* the distribution set ID
*
* @return the number of software modules matching the given distribution
* set ID.
*/
long countByAssignedToId(Long setId);
/**
* @param pageable
* the page request to page the result set

View File

@@ -65,9 +65,10 @@ public class AutoAssignScheduler {
LOGGER.debug("auto assign schedule checker has been triggered.");
// run this code in system code privileged to have the necessary
// permission to query and create entities.
systemSecurityContext.runAsSystem(() -> executeAutoAssign());
systemSecurityContext.runAsSystem(this::executeAutoAssign);
}
@SuppressWarnings("squid:S3516")
private Object executeAutoAssign() {
// workaround eclipselink that is currently not possible to
// execute a query without multitenancy if MultiTenant

View File

@@ -216,7 +216,7 @@ public final class RSQLUtility {
private final SimpleTypeConverter simpleTypeConverter;
final Database database;
private final Database database;
private JpqQueryRSQLVisitor(final Root<T> root, final CriteriaBuilder cb, final Class<A> enumType,
final VirtualPropertyReplacer virtualPropertyReplacer, final Database database) {

View File

@@ -0,0 +1,135 @@
/**
* Copyright (c) 2018 Bosch Software Innovations GmbH and others.
*
* All rights reserved. This program and the accompanying materials
* are made available under the terms of the Eclipse Public License v1.0
* which accompanies this distribution, and is available at
* http://www.eclipse.org/legal/epl-v10.html
*/
package org.eclipse.hawkbit.repository.jpa.utils;
import java.util.function.Function;
import javax.validation.constraints.NotNull;
import org.eclipse.hawkbit.repository.exception.QuotaExceededException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.validation.annotation.Validated;
/**
* Helper class to check assignment quotas.
*/
@Validated
public final class QuotaHelper {
/**
* Class logger
*/
private static final Logger LOG = LoggerFactory.getLogger(QuotaHelper.class);
private QuotaHelper() {
// no need to instantiate this class
}
/**
* Asserts the specified assignment quota.
*
* @param requested
* The number of entities that shall be assigned to the parent
* entity.
* @param limit
* The maximum number of entities that may be assigned to the
* parent entity.
* @param type
* The type of the entities that shall be assigned.
* @param parentType
* The type of the parent entity.
*
* @throws QuotaExceededException
* if the assignment operation would cause the quota to be
* exceeded
*/
public static void assertAssignmentQuota(final long requested, final long limit, @NotNull final Class<?> type,
@NotNull final Class<?> parentType) {
assertAssignmentQuota(null, requested, limit, type.getSimpleName(), parentType.getSimpleName(), null);
}
/**
* Asserts the specified assignment quota.
*
* @param parentId
* The ID of the parent entity.
* @param requested
* The number of entities that shall be assigned to the parent
* entity.
* @param limit
* The maximum number of entities that may be assigned to the
* parent entity.
* @param type
* The type of the entities that shall be assigned.
* @param parentType
* The type of the parent entity.
* @param countFct
* Function to count the entities that are currently assigned to
* the parent entity.
*
* @throws QuotaExceededException
* if the assignment operation would cause the quota to be
* exceeded
*/
public static void assertAssignmentQuota(final Long parentId, final long requested, final long limit,
@NotNull final Class<?> type, @NotNull final Class<?> parentType, final Function<Long, Long> countFct) {
assertAssignmentQuota(parentId, requested, limit, type.getSimpleName(), parentType.getSimpleName(), countFct);
}
/**
* Asserts the specified assignment quota.
*
* @param parentId
* The ID of the parent entity.
* @param requested
* The number of entities that shall be assigned to the parent
* entity.
* @param limit
* The maximum number of entities that may be assigned to the
* parent entity.
* @param type
* The type of the entities that shall be assigned.
* @param parentType
* The type of the parent entity.
* @param countFct
* Function to count the entities that are currently assigned to
* the parent entity.
*
* @throws QuotaExceededException
* if the assignment operation would cause the quota to be
* exceeded
*/
public static void assertAssignmentQuota(final Long parentId, final long requested, final long limit,
@NotNull final String type, @NotNull final String parentType, final Function<Long, Long> countFct) {
// check if the quota is unlimited
if (limit <= 0) {
LOG.debug("Quota 'Max {} entities per {}' is unlimited.", type, parentType);
return;
}
if (requested > limit) {
final String parentIdStr = parentId != null ? String.valueOf(parentId) : "<new>";
LOG.warn("Cannot assign {} {} entities to {} '{}' because of the configured quota limit {}.", requested,
type, parentType, parentIdStr, limit);
throw new QuotaExceededException(type, parentType, parentId, requested, limit);
}
if (parentId != null && countFct != null) {
final long currentCount = countFct.apply(parentId);
if (currentCount + requested > limit) {
LOG.warn(
"Cannot assign {} {} entities to {} '{}' because of the configured quota limit {}. Currently, there are {} {} entities assigned.",
requested, type, parentType, parentId, limit, currentCount, type);
throw new QuotaExceededException(type, parentType, parentId, requested, limit);
}
}
}
}