UX evaluation of Run analysis_AIP Console

“A problem well stated is a problem half-solved.”-Charles Kettering

7 min readApr 9, 2021

Context

CAST is a software intelligence company for the top digital leaders in the world. (To know more about CAST, click here).

It believes in investing in the user experience and making them better in the process.

One of the top aspect of the company’s design process is to have critical UX analysis of even the tiniest functional features of the product following the Lean design approach.

By stepping into the user’s shoes through the analysis, CAST tries to improve and better the product purely from the user’s perspective.

Here is one of the design story of a small feature “ Run analysis dialogue box” from the AIP Console product. The users of this product are dedicated developers who are building data for consumers via Console. The operator (developers) receives application code, does some basic configuration, and run analysis/take snapshots. The super operator (developers) does advanced configuration like RF, DLM, user-defined modules, transaction configuration.(To know more about AIP Console, contact us!)

Once you parsed the source code and created objects and links, then you can measure the quality by checking quality rules. This will produce results in the dashboard. this set of results is called a snapshot. it is a kind of “picture” of the application at a given time.

Improving the way to process and publish analyzed results in the so called “Run analysis” dialogue box — a UX case study.

The challenge/problem
The dialogue box is quite confusing and it’s not clear for the user that what’s happening and how to accomplish the tasks effectively and efficiently.

Objective

The objective of this dialogue box is to run the analysis following a certain number of steps, depending on the product/tool where we want to process/publish the results.

For example:

If we want to publish the result in Imaging/Architecture studio, we just want to take the step “Run analysis.

If we want to publish the result in Engineering Dashboard, we need to take the first two steps 1)Run Analysis and 2) Take the snapshot

If we want to publish the result in Health Dashboard, we need to take all the three steps as in 1) Run the analysis, 2) Take the snapshot and then 3) Publish it.

When we click on the green icon (in the bottom right corner Image (a)), the Run analysis dialogue box appears as shown below in the Image (b).

As we see, these are the following elements in the Run analysis dialogue box:

Application version- Allowing the user to select different versions which are in different states.
Back up application- Allowing the user to keep the backup of application, so as to have original source code/app safe in case we need it. Otherwise, it’s not possible to revert it back after running the analysis. There could be changes in the source code or configuration of the app after running the analysis. But it’s not mandatory to save it if we want to update the original data.
Set as current version- We ask user to set the version as the “current version” as analysis can be done only on current version.
Run analysis- If user wants to just take the action of “Run analysis”, the result would be processed and published by default in Architecture studio, Transaction and modules. The data will be saved in the local schema. (In Local schema, the data can be saved after analysis. It can only save current analyzed version. If we want to save analysed data of another version, the previous data will be overwritten.
Take snapshot- If user wants to just take the action of “Taking the snapshot” after running the analysis. The result will be published by default to Central schema i.e. Engineering Dashboard. (In the central schema, all the analyzed data of snapshot gets saved).
Publish to Health Dashboard-If the user wants to publish the current snapshot data to the Health Dashboard, the data will be saved in the Measurement database. (Measurement database contains the snapshots for multiple applications and presents views for management team)
Redo the actions-We can rerun the analysis and redo the action for “Take snapshot”, if we are not satisfied with the result or if we changed the configuration of the source code.

Here is the UX evaluation of the existing dialogue box for Run analysis (step by step):

1. Application version (Image 1)

Non-standard approach to Interaction rule “Hick’s law”.
Analysis can be done only on “Current version”. By providing number of version choices, we are increasing the decision time for the user.

How would a user know the status of version as it’s not mentioned along with the version name?

Recommendation(s): We should just show the current version rather than asking user to choose, hence making the process efficient. (The version status setting can be done in the relatable place, maybe in the version.

2. Back up application (Image 2)

Non-standard approach to Usability heuristic “User control and freedom”
The interface should ensure that the user never lose their data and moreover they are aware of the action and consequences of it.
What’s the need of back up?- (Explained above) As it’s not a mandatory action, but it’s an important action. In that case, we can’t irritate the user or consume the time and space by backing up the app every time. On the other hand, we also want user to not forget backing up the application as it will be impossible to revert.

Recommendation(s): Hence the solution could be, Allowing the user to decide whether he wants to “Back up” or not. Also informing them about the the importance of it via tooltip.

3. Set as current version (Image 3)

Non-standard approach to usability heuristic “Aesthetic and minimalist design”
Why do we need to add an extra step of asking the user to “set it as current version” first, while we know that analysis can run only on the current version.

Recommendation(s): We can show only the current version in the Application version section above and remove the step of asking the user to “set it as a current version”, hence decreasing the cognitive load on the user.

4. Inappropriate grouping (Image 4)

Non-standard approach to Interaction rule “Law of proximity” and “Fitt’s law”
Image 4(a)-Visually, it looks like “Back up application is grouped together with three sub-tasks, while that’s not the case.
The empty circle gives an impression of radio button while it’s not the case either.

Recommendation(s): Based on the actions and relativity, grouping should be done.

Image 4(b)-The eye needs to travel a distance to understand that estimated run time has been given for the specific actions above like Run analysis and Take a snapshot. The user might not even observe it.

Recommendation(s): Knowing the “estimated time” at the time of action would keep the user informed and would end up helping user in the decision. Hence it should be placed next to the actions.

5. Run Analysis and Take a snapshot (Image 5)

Non-standard approach to usability heuristics “Recognition rather than recall”
The user doesn’t know the reason behind this step. There is also a call to action button at the end with almost same name which might create confusion.
The interface should not demand user to remember information for taking an action. Rather the interface should make user aware/educate about the step to make the decision easier.

Recommendation(s): The user might not want to read the information about it always whenever he has to take this action. But we can make the information available when required by placing “an info icon button” next to it. On hover, the user should be able to get info about the goal of this action.

6. Redo action

Non-standard approach to Usability evaluation “Visibility of system status”, “Match between system and the real world”, Recognition rather than recall”
Redo action (By double checking the already checked circle) is not obvious to the user and there are chances of user getting lost.

Recommendation(s): Use the familiar way of restarting the action.