Protection against misuse with system-wide quota definitions (#670)

* Added quota for meta data per software module

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Added unit test for "meta data per module" quota

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* quota test enhancements

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Verify enforcement of meta data quota via REST

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quota for distribution set meta data

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Verify enforcement of distribution set meta data quota via REST

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* software modules per distribution set quota

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Integration test enhancements for Modules per DistSet quota

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quota for software module types per distribution set type

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quota for max artifacts per software module

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quotas for ActionStatus per Action and Messages per ActionStatus

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quota attributes per target

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quota targets per rollout group

Signed-off-by: stefbehl <stefan.behl@bosch-si.com>

* Quota max targets per rollout group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per rollout group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per rollout group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per rollout group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* quota max actions per target

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per rollout group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max actions per target

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max actions per target

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per auto assignment

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per manual assignment

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per auto assignment

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per auto assign

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per auto assignment

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max actions per target

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per manual assignment

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix issues caused by merge

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Improve JavaDoc

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Sonar issues

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Sonar findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max artifact size

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Optimize quota configuration

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix test failures

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Improve test coverage

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max rollout groups per rollout

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Configure Rollout UI enhancements

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* UI enhancements

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Minor changes

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Quota max targets per group

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Sonar findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Sonar findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Sonar finding

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix code review findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix review findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* New approach for 'max artifact size' enforcement

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Sonar findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix failing tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Reduce max artifact size for tests

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>

* Fix Kai's review findings

Signed-off-by: Stefan Behl <stefan.behl@bosch-si.com>
This commit is contained in:
Stefan Behl
2018-05-02 12:09:29 +02:00
committed by Kai Zimmermann
parent fcc15a0484
commit 6dd98d2134
63 changed files with 2383 additions and 523 deletions

View File

@@ -60,8 +60,6 @@ import ru.yandex.qatools.allure.annotations.Stories;
@SpringApplicationConfiguration(classes = DownloadTestConfiguration.class)
public class DdiArtifactDownloadTest extends AbstractDDiApiIntegrationTest {
private static final int ARTIFACT_SIZE = 5 * 1024 * 1024;
private static volatile int downLoadProgress = 0;
private static volatile long shippedBytes = 0;
@@ -84,9 +82,10 @@ public class DdiArtifactDownloadTest extends AbstractDDiApiIntegrationTest {
assignDistributionSet(ds, targets);
// create artifact
final byte random[] = RandomUtils.nextBytes(5 * 1024);
final int artifactSize = 5 * 1024;
final byte random[] = RandomUtils.nextBytes(artifactSize);
final Artifact artifact = artifactManagement.create(new ByteArrayInputStream(random),
ds.findFirstModuleByType(osType).get().getId(), "file1", false);
ds.findFirstModuleByType(osType).get().getId(), "file1", false, artifactSize);
// no artifact available
mvc.perform(get("/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/123455",
@@ -170,9 +169,10 @@ public class DdiArtifactDownloadTest extends AbstractDDiApiIntegrationTest {
final DistributionSet ds = testdataFactory.createDistributionSet("");
// create artifact
final byte random[] = RandomUtils.nextBytes(ARTIFACT_SIZE);
final int artifactSize = 5 * 1024 * 1024;
final byte random[] = RandomUtils.nextBytes(artifactSize);
final Artifact artifact = artifactManagement.create(new ByteArrayInputStream(random),
ds.findFirstModuleByType(osType).get().getId(), "file1", false);
ds.findFirstModuleByType(osType).get().getId(), "file1", false, artifactSize);
// download fails as artifact is not yet assigned
mvc.perform(get("/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}",
@@ -180,11 +180,9 @@ public class DdiArtifactDownloadTest extends AbstractDDiApiIntegrationTest {
// now assign and download successful
assignDistributionSet(ds, targets);
final MvcResult result = mvc
.perform(
get("/{tenant}/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}",
tenantAware.getCurrentTenant(), target.getControllerId(), getOsModule(ds),
artifact.getFilename()))
final MvcResult result = mvc.perform(get(
"/{tenant}/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}",
tenantAware.getCurrentTenant(), target.getControllerId(), getOsModule(ds), artifact.getFilename()))
.andExpect(status().isOk()).andExpect(content().contentType(MediaType.APPLICATION_OCTET_STREAM))
.andExpect(header().string("Accept-Ranges", "bytes"))
.andExpect(header().string("Last-Modified", dateFormat.format(new Date(artifact.getCreatedAt()))))
@@ -196,7 +194,7 @@ public class DdiArtifactDownloadTest extends AbstractDDiApiIntegrationTest {
// download complete
assertThat(downLoadProgress).isEqualTo(10);
assertThat(shippedBytes).isEqualTo(ARTIFACT_SIZE);
assertThat(shippedBytes).isEqualTo(artifactSize);
}
@Test
@@ -211,16 +209,15 @@ public class DdiArtifactDownloadTest extends AbstractDDiApiIntegrationTest {
assignDistributionSet(ds, target);
// create artifact
final byte random[] = RandomUtils.nextBytes(5 * 1024);
final int artifactSize = 5 * 1024;
final byte random[] = RandomUtils.nextBytes(artifactSize);
final Artifact artifact = artifactManagement.create(new ByteArrayInputStream(random), getOsModule(ds), "file1",
false);
false, artifactSize);
// download
final MvcResult result = mvc
.perform(
get("/{tenant}/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}.MD5SUM",
tenantAware.getCurrentTenant(), target.getControllerId(), getOsModule(ds),
artifact.getFilename()))
final MvcResult result = mvc.perform(get(
"/{tenant}/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}.MD5SUM",
tenantAware.getCurrentTenant(), target.getControllerId(), getOsModule(ds), artifact.getFilename()))
.andExpect(status().isOk()).andExpect(header().string("Content-Disposition",
"attachment;filename=" + artifact.getFilename() + ".MD5SUM"))
.andReturn();
@@ -245,7 +242,7 @@ public class DdiArtifactDownloadTest extends AbstractDDiApiIntegrationTest {
// create artifact
final byte random[] = RandomUtils.nextBytes(resultLength);
final Artifact artifact = artifactManagement.create(new ByteArrayInputStream(random), getOsModule(ds), "file1",
false);
false, resultLength);
assertThat(random.length).isEqualTo(resultLength);
@@ -259,11 +256,10 @@ public class DdiArtifactDownloadTest extends AbstractDDiApiIntegrationTest {
for (int i = 0; i < resultLength / range; i++) {
final String rangeString = "" + i * range + "-" + ((i + 1) * range - 1);
final MvcResult result = mvc
.perform(
get("/{tenant}/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}",
tenantAware.getCurrentTenant(), target.getControllerId(), getOsModule(ds), "file1")
.header("Range", "bytes=" + rangeString))
final MvcResult result = mvc.perform(get(
"/{tenant}/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}",
tenantAware.getCurrentTenant(), target.getControllerId(), getOsModule(ds), "file1").header("Range",
"bytes=" + rangeString))
.andExpect(status().isPartialContent()).andExpect(header().string("ETag", artifact.getSha1Hash()))
.andExpect(content().contentType(MediaType.APPLICATION_OCTET_STREAM))
.andExpect(header().string("Accept-Ranges", "bytes"))
@@ -278,11 +274,10 @@ public class DdiArtifactDownloadTest extends AbstractDDiApiIntegrationTest {
assertThat(outputStream.toByteArray()).isEqualTo(random);
// return last 1000 Bytes
MvcResult result = mvc
.perform(
get("/{tenant}/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}",
tenantAware.getCurrentTenant(), target.getControllerId(), getOsModule(ds), "file1")
.header("Range", "bytes=-1000"))
MvcResult result = mvc.perform(
get("/{tenant}/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}",
tenantAware.getCurrentTenant(), target.getControllerId(), getOsModule(ds), "file1")
.header("Range", "bytes=-1000"))
.andExpect(status().isPartialContent()).andExpect(header().string("ETag", artifact.getSha1Hash()))
.andExpect(content().contentType(MediaType.APPLICATION_OCTET_STREAM))
.andExpect(header().string("Accept-Ranges", "bytes"))
@@ -296,11 +291,10 @@ public class DdiArtifactDownloadTest extends AbstractDDiApiIntegrationTest {
.isEqualTo(Arrays.copyOfRange(random, resultLength - 1000, resultLength));
// skip first 1000 Bytes and return the rest
result = mvc
.perform(
get("/{tenant}/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}",
tenantAware.getCurrentTenant(), target.getControllerId(), getOsModule(ds), "file1")
.header("Range", "bytes=1000-"))
result = mvc.perform(
get("/{tenant}/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}",
tenantAware.getCurrentTenant(), target.getControllerId(), getOsModule(ds), "file1")
.header("Range", "bytes=1000-"))
.andExpect(status().isPartialContent()).andExpect(header().string("ETag", artifact.getSha1Hash()))
.andExpect(content().contentType(MediaType.APPLICATION_OCTET_STREAM))
.andExpect(header().string("Accept-Ranges", "bytes"))
@@ -326,11 +320,10 @@ public class DdiArtifactDownloadTest extends AbstractDDiApiIntegrationTest {
.andExpect(header().string("Content-Disposition", "attachment;filename=file1"));
// multipart download - first 20 bytes in 2 parts
result = mvc
.perform(
get("/{tenant}/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}",
tenantAware.getCurrentTenant(), target.getControllerId(), getOsModule(ds), "file1")
.header("Range", "bytes=0-9,10-19"))
result = mvc.perform(
get("/{tenant}/controller/v1/{controllerId}/softwaremodules/{softwareModuleId}/artifacts/{filename}",
tenantAware.getCurrentTenant(), target.getControllerId(), getOsModule(ds), "file1")
.header("Range", "bytes=0-9,10-19"))
.andExpect(status().isPartialContent()).andExpect(header().string("ETag", artifact.getSha1Hash()))
.andExpect(content().contentType("multipart/byteranges; boundary=THIS_STRING_SEPARATES_MULTIPART"))
.andExpect(header().string("Accept-Ranges", "bytes"))

View File

@@ -21,6 +21,8 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
import java.util.HashMap;
import java.util.Map;
import org.eclipse.hawkbit.exception.SpServerError;
import org.eclipse.hawkbit.repository.exception.QuotaExceededException;
import org.eclipse.hawkbit.repository.model.Target;
import org.eclipse.hawkbit.rest.util.JsonBuilder;
import org.eclipse.hawkbit.rest.util.MockMvcResultPrinter;
@@ -109,13 +111,13 @@ public class DdiConfigDataTest extends AbstractDDiApiIntegrationTest {
@Test
@Description("We verify that the config data (i.e. device attributes like serial number, hardware revision etc.) "
+ "upload limitation is inplace which is meant to protect the server from malicious attempts.")
public void putToMuchConfigData() throws Exception {
+ "upload quota is enforced to protect the server from malicious attempts.")
public void putTooMuchConfigData() throws Exception {
testdataFactory.createTarget("4717");
// initial
Map<String, String> attributes = new HashMap<>();
for (int i = 0; i < 10; i++) {
for (int i = 0; i < quotaManagement.getMaxAttributeEntriesPerTarget(); i++) {
attributes.put("dsafsdf" + i, "sdsds" + i);
}
mvc.perform(put("/{tenant}/controller/v1/4717/configData", tenantAware.getCurrentTenant())
@@ -126,7 +128,9 @@ public class DdiConfigDataTest extends AbstractDDiApiIntegrationTest {
attributes.put("on too many", "sdsds");
mvc.perform(put("/{tenant}/controller/v1/4717/configData", tenantAware.getCurrentTenant())
.content(JsonBuilder.configData("", attributes, "closed")).contentType(MediaType.APPLICATION_JSON))
.andExpect(status().isForbidden());
.andExpect(status().isForbidden())
.andExpect(jsonPath("$.exceptionClass", equalTo(QuotaExceededException.class.getName())))
.andExpect(jsonPath("$.errorCode", equalTo(SpServerError.SP_QUOTA_EXCEEDED.getKey())));
}
@@ -192,8 +196,7 @@ public class DdiConfigDataTest extends AbstractDDiApiIntegrationTest {
}
@Step
private void putConfigDataWithInvalidUpdateMode(final String configDataPath)
throws Exception {
private void putConfigDataWithInvalidUpdateMode(final String configDataPath) throws Exception {
// create some attriutes
final Map<String, String> attributes = new HashMap<>();

View File

@@ -110,11 +110,12 @@ public class DdiDeploymentBaseTest extends AbstractDDiApiIntegrationTest {
final DistributionSet ds = testdataFactory.createDistributionSet("", true);
final DistributionSet ds2 = testdataFactory.createDistributionSet("2", true);
final byte random[] = RandomUtils.nextBytes(5 * 1024);
final int artifactSize = 5 * 1024;
final byte random[] = RandomUtils.nextBytes(artifactSize);
final Artifact artifact = artifactManagement.create(new ByteArrayInputStream(random), getOsModule(ds), "test1",
false);
false, artifactSize);
final Artifact artifactSignature = artifactManagement.create(new ByteArrayInputStream(random), getOsModule(ds),
"test1.signature", false);
"test1.signature", false, artifactSize);
final Target savedTarget = testdataFactory.createTarget("4712");
@@ -198,10 +199,9 @@ public class DdiDeploymentBaseTest extends AbstractDDiApiIntegrationTest {
.andExpect(jsonPath("$.deployment.chunks[?(@.part==os)].artifacts[1].hashes.sha1",
contains(artifactSignature.getSha1Hash())))
.andExpect(
jsonPath("$.deployment.chunks[?(@.part==os)].artifacts[1]._links.download-http.href",
contains(HTTP_LOCALHOST + tenantAware.getCurrentTenant()
+ "/controller/v1/4712/softwaremodules/"
.andExpect(jsonPath("$.deployment.chunks[?(@.part==os)].artifacts[1]._links.download-http.href",
contains(
HTTP_LOCALHOST + tenantAware.getCurrentTenant() + "/controller/v1/4712/softwaremodules/"
+ findDistributionSetByAction.findFirstModuleByType(osType).get().getId()
+ "/artifacts/test1.signature")))
.andExpect(jsonPath("$.deployment.chunks[?(@.part==os)].artifacts[1]._links.md5sum-http.href",
@@ -268,11 +268,12 @@ public class DdiDeploymentBaseTest extends AbstractDDiApiIntegrationTest {
final String visibleMetadataOsKey = "metaDataVisible";
final String visibleMetadataOsValue = "withValue";
final byte random[] = RandomUtils.nextBytes(5 * 1024);
final int artifactSize = 5 * 1024;
final byte random[] = RandomUtils.nextBytes(artifactSize);
final Artifact artifact = artifactManagement.create(new ByteArrayInputStream(random), getOsModule(ds), "test1",
false);
false, artifactSize);
final Artifact artifactSignature = artifactManagement.create(new ByteArrayInputStream(random), getOsModule(ds),
"test1.signature", false);
"test1.signature", false, artifactSize);
softwareModuleManagement.createMetaData(entityFactory.softwareModuleMetadata().create(getOsModule(ds))
.key(visibleMetadataOsKey).value(visibleMetadataOsValue).targetVisible(true));
@@ -391,11 +392,12 @@ public class DdiDeploymentBaseTest extends AbstractDDiApiIntegrationTest {
final DistributionSet ds = testdataFactory.createDistributionSet("", true);
final DistributionSet ds2 = testdataFactory.createDistributionSet("2", true);
final byte random[] = RandomUtils.nextBytes(5 * 1024);
final int artifactSize = 5 * 1024;
final byte random[] = RandomUtils.nextBytes(artifactSize);
final Artifact artifact = artifactManagement.create(new ByteArrayInputStream(random), getOsModule(ds), "test1",
false);
false, artifactSize);
final Artifact artifactSignature = artifactManagement.create(new ByteArrayInputStream(random), getOsModule(ds),
"test1.signature", false);
"test1.signature", false, artifactSize);
final Target savedTarget = testdataFactory.createTarget("4712");