How to kill many birds with one stone? – Automated modification of hundreds of Adobe Analysis Workspace Projects
on 28.07.2021 by FELD M Data Product Team
As a consequence of a client’s legal decision, we were tasked to add disclaimers to hundreds of Adobe Analysis Workspace Projects. The following requirements had to be met:
- For each project, a new panel including the disclaimer must be positioned at the top, prior to any other report element.
- The report suite for the disclaimer panel should be replicated from the second panel.
- A tag stating that a disclaimer was included in the workspace was to be added.
Adapting numerous projects manually would have been very time consuming and cause high potential for error. To avoid this, we looked for an option to adapt the reports via API.
The Solution.
A Repository with Focus on the Endpoint.
In addition to the Adobe Analytics API 1.4, we are familiar with Adobe Analytics API 2.0 and have used both in several projects. Neither version provide the possibility to create or modify Workspaces. Just before we thought we had to bite the bullet and adjust all Adobe Analysis Workspace Projects manually, we stumbled across a repository, which includes the method “updateProject”. The author of this work, Julien Piccini, dedicated special attention to the project endpoint and crafted a separate readme. (Julien, should you read this post, thanks a lot for your awesome work!)
The main takeaways for our use case were as follows:
- `getProject` and `getProjects` methods are BETA and may stop working at any time
- The library does not work for all Workspaces as the internal structure of Workspaces has evolved over time
- The `updateProject` expects the entire definition of the Workspace and performs a PUT operation meaning that it entirely replaces the existing content
Getting started.
As soon as we found the method, we started experimenting with our Adobe Analytics Sandbox to test it for our use case. To run the first tests, we started building our own python client focusing on the `project` endpoint and an OAuth integration. We decided against a JWT integration because this would have required additional alignment with the client. After some coding, we had everything in place to fetch some example Workspaces to further examine their structure.
While the library parses the results, a series of internal checks take place, which ensure the compatibility of the Workspace with the library requirements. Unfortunately, this method did not work with older Workspaces, however we worked around this by using a plain JSON instead of the library parsing the results, which ensured the validity of the returned endpoints.
Here you can see the structure of a simple Workspace showing our desired disclaimer:
{
'accessLevel': 'edit',
'approved': False,
'companyTemplate': False,
'created': '2021-05-21T10:00:00Z',
'definition': {
// We'll have a look into this later
},
'description': '<PROJECT DESCRIPTION>',
'favorite': False,
'id': '<PROJECT ID>',
'modified': '2021-05-21T11:00:00Z',
'name': '<PROJECT NAME>',
'owner': {
'id': -1,
'login': 'XXX',
'name': 'XXX'
},
'reportSuiteName': '<REPORTSUITE NAME>',
'rsid': '<REPORTSUITE ID>',
'shares': [{
'accessLevel': 'edit',
'componentId': '<PROJECT ID>',
'componentType': 'project',
'shareId': -1,
'shareToDisplayName': 'Firstname Lastname',
'shareToId': -1,
'shareToLogin': 'firstname.lastname@example.com',
'shareToType': 'user'
}],
'siteTitle': '<SITE TITLE>',
'tags': [],
'type': 'project'
}
Reviewing our test project’s structure, we identified our two major points of action, namely `definition` and `tags`. `definition` contains the project setup itself and `tags` to add a tag to each report to keep track of which reports have already been modified via the API.
Definition and Version.
Here you can see the `definition` of the Project:
'definition': {
'additionalCuratedComponents': [],
'countRepeatInstances': True,
'currentWorkspaceIndex': 0,
'customColorSchemes': [],
'isCurated': False,
'version': '28',
'viewDensity': 'expanded',
'workspaces': []
}
One important aspect in the project definition is the `version` attribute which denotes Adobe’s version of the Workspace. According to our tests, there was a major change between version 20 and 21, which we understand to be the reason that “The definition are not consistent over time, therefore the Project class may not work on old projects” [Source] as stated by the library’s author. To be on the safe side, we focused on projects with a `version` >= 21, which in our case was luckily most of the projects that we had to modify.
We were surprised by the presence of a list of Workspaces, although all the Workspaces we manually screened always only had one element. For the further automisation, checks were introduced to our code to stop the process if the list length did not equal one. However, this was never necessary.
An element of a Workspace is a simple JSON storing the corresponding `id`, `name` and `panels`. The `panels` element within the `workspace` contains the actual elements of the project.
Below you can see an example of the disclaimer panel we had to add:
{
'annotations': [],
'collapsed': False,
'dateRange': {
'id': 'thisMonth',
'__entity__': True,
'type': 'DateRange',
'__metaData__': {
'name': 'This month'
}
},
'description': '',
'id': 'BBBBBBBB-BBBB-BBBB-BBBB-BBBBBBBBBBBB',
'name': '<PANEL NAME>',
'position': {
'autoHeight': 200,
'autoSize': True,
'width': 100,
'x': 0,
'y': 0
},
'reportSuite': {
'id': '<REPORTSUITE ID>',
'__entity__': True,
'type': 'ReportSuite',
'__metaData__': {
'name': '<REPORTSUITE NAME>',
'rsid': '<REPORTSUITE ID>'
}
},
'segmentGroups': [],
'subPanels': [
{
'collapsed': False,
'description': '<SUBPANEL DESCRIPTION>',
'id': 'CCCCCCCC-CCCC-CCCC-CCCC-CCCCCCCCCCCC',
'isQuickInsightsSubPanel': False,
'linkedSourceId': '',
'name': '<SUBPANEL NAME>',
'position': {
'autoHeight': 127,
'autoSize': True,
'width': 100,
'x': 0,
'y': 0},
'reportlet': {
'isConfigVisible': True,
'name': '<REPORTLET NAME>',
'showVizSelectorOnSubPanel': False,
'textContent': '{"ops":[{"insert":"<HERE GOES OUR DISCLAIMER>.\\n"}]}',
'type': 'TextReportlet',
'useRowBasedPercentages': False
},
'swatchColor': '#BB0A30',
'type': 'genericSubPanel',
'visible': True,
'visualizationIndex': 1
}
],
'type': 'panel'
}
Testing, Testing, 1, 2, 3.
As the data proved easy to map against the information shown in the User Interface (Report Suite information, Panel Names, etc), we felt confident to continue following the strategy to modify the Workspaces via API. In our first test, we added a disclaimer to a test Workspace project (by simply adding the data shown above to the `panels` list) to further examine effects on the structure. This first step showed the following results:
- No extensive validation on Adobe side as UUIDs were used irrespective of whether they were duplicated.
- The disclaimer panels in the Workspace UI appeared well, despite the two disclaimer panels coordinates overlapping.
As these attributes can easily be calculated and generated, we decided to modify them by dynamically calculating the positions (by adding the size of the disclaimer to the `y` coordinate of the following panels or generating UUIDs for the disclaimer).
After these adjustments were coded, we tested to see if the solution met the prerequisites:
- For each project, a new panel including the disclaimer must be positioned at the top, prior to any other report element.
- The report suite for the disclaimer panel should be replicated from the second panel.
- A tag stating that a disclaimer was included in the workspace was to be added.
Luckily, it worked smoothly with only one exception; our assigned tags were not attached. We had tested different combinations of how to add tags to the project.
Looking at the other (non BETA) API endpoints, the `PUT` method for e.g. Segments states that tags are not processed, so we assumed that this is also the case for the Project endpoint.
This requires a second step to assign tags after updating the Workspace itself, luckily tagging is well documented by Adobe, so we were able to add them accordingly.
In a Nutshell: Which Steps must be Taken?
The following gives an overview of the process:
- Create a Workspace containing the disclaimer acting as a template that should be added to existing Workspace projects
- Iterate through the Projects that should be updated
- Dump all information that can be gathered via API for the Project
- Check preconditions (Tag already exists, Disclaimer already exists, Workspace version, …)
- Regenerate UUIDs for the disclaimer
- Create a new list of panels with the disclaimer in the first position, followed by all existing panels and assign it appropriately to the definition
- Adjust coordinates for the existing panels
- Perform PUT operation on Project endpoint
- Dump the payload that has been sent to the Project endpoint
- Assign an appropriate tag to the updated project
The process went very smoothly without a single error and all Workspace projects looked as expected.
Main takeaways.
- There is an unofficial Project endpoint in Adobe 2.0 APIs to modify Workspace projects
- Not all Workspace projects are suitable to be modified via API (According to our data Workspace projects with version >= 21 are the exception)
- On the fly tagging is not possible, as with other API endpoints
- No extensive validation takes place on the Adobe side (e. g. duplicate IDs, colliding coordinates)