Exploring Product Units, Stable ConvPU and Semantic Engineering in Medical Image Segmentation
In our previous blog 'AI Breast Cancer Detection: How Segmentation Models Read What Scans Miss,' we looked at why identifying a suspicious area in a breast ultrasound image is only the first step.
Accurate tumor localization and segmentation are also important because they define where the tumor is and how closely the predicted boundary matches an expert annotation.
This raises another question: Can a model learn how different tumor features relate to one another?
Breast ultrasound images present several challenges when segmenting tumors, including noise, low contrast, irregular tumor boundaries and variation in tumor appearance. A tumor may be characterized by several features at once, including texture, shape, boundary, contrast and surrounding context.
Maya and Arjun, two AI engineers, explored this question through Semantic Engineering and Product Units. Their work led to StableConvPU, a more stable approach to multiplicative feature learning, and eventually to PU-UNet, a U-Net architecture that incorporates Product Units into deeper layers.
This article follows that research journey and examines how PU-UNet was tested on the BUSI breast ultrasound dataset.
Late one evening in an AI innovation lab, Maya was reviewing a breast ultrasound image.
Maya: "We have trained many segmentation models. Accuracy keeps improving, but I keep thinking about the relationships between the features the model is learning."
Arjun: "What makes those relationships important?"
Maya: "A tumor is not defined by one characteristic. Texture, shape, boundary and contrast all contribute to how the lesion appears in an image."
That question became the starting point for the research.
Medical image segmentation works at the pixel level. The model assigns a class to each pixel and produces a mask that outlines the region of interest. For breast ultrasound, that means identifying the tumor and defining its boundary within the surrounding tissue.
The distinction between detection and segmentation is important.
A classification model can answer a question such as:
Is a tumor present?
A segmentation model addresses a different question:
Where exactly is the tumor?
The segmentation output is a pixel-level mask that outlines the tumor boundary.
Maya: "So identifying a tumor and defining its boundary are separate problems."
Arjun: "Exactly. Localization gives us the spatial information that a classification result does not provide."
For a detailed discussion of breast cancer detection and the role of localization, see AI Breast Cancer Detection: How Segmentation Models Read What Scans Miss.
U-Net is widely used for medical image segmentation because its architecture combines feature extraction with spatial information needed to reconstruct detailed segmentation masks.
Maya: "Most teams start with U-Net, right?"
Arjun: "For good reason."
The encoder extracts increasingly abstract features from the input image. The decoder uses those features to reconstruct the segmentation mask. Skip connections carry spatial information from earlier layers into the decoder, helping preserve details that are important for defining boundaries.
U-Net provides a strong foundation for segmentation. The research question was whether the way features are combined inside the network could be improved for difficult breast ultrasound images.
The next question focused on how conventional neural networks combine information.
In a conventional convolutional neural network, features are combined through learned weighted operations. A simplified representation is:
Texture + Shape + Contrast + Boundary
y = w₁x₁ + w₂x₂ + w₃x₃ + b
This approach allows the network to accumulate information from multiple features.
Maya and Arjun wanted to explore a different mathematical relationship.
Maya: "What if the model could learn how the features interact, instead of treating each feature as separate evidence?"
This is where Semantic Engineering became relevant to the research.
Semantic Engineering focuses on relationships between information. In this context, the research explored whether relationships among texture, shape, boundary and contrast could be represented directly within the model.
A tumor may exhibit several characteristics together:
The interaction between these characteristics can provide useful information for segmentation in medical imaging.
The research therefore turned to Product Units, which use multiplicative relationships between features.
Maya: "What if we teach the model to represent those relationships directly?"
Arjun: "That is where Product Units come in."
A simple example illustrates the difference.
A conventional additive combination might look like:
0.8 + 0.7 + 0.9 = 2.4
A Product Unit combines the same values multiplicatively:
0.8 × 0.7 × 0.9 = 0.504
Now consider what happens when one feature becomes much weaker:
0.8 × 0.1 × 0.9 = 0.072
The combined value drops significantly because one input has changed substantially.
This provides a way to represent interactions between features such as texture, shape and tumor boundaries, where a change in one feature can affect the combined result.
For the breast ultrasound segmentation problem, this raised an interesting possibility: could multiplicative feature relationships help the model distinguish tumor structures from surrounding tissue?
There was, however, a technical challenge.
The first implementation exposed a problem with multiplicative learning.
Maya: "What happened?"
Arjun: "The multiplication is powerful, but the numerical behavior is difficult to control."
Multiplication can create unstable values during training. For example:
log(0)
is undefined, while:
exp(100)
produces an extremely large value.
These behaviors can cause numerical problems during model training.
The team therefore developed StableConvPU, a sequence designed to make multiplicative learning more stable:
Softplus → Log → Conv → Clip → Exp
Each stage has a specific role:
After addressing the numerical challenge, the team could examine where Product Units should be placed within U-Net.
The next step was to bring Product Units into the U-Net architecture.
Before training, the ultrasound images needed a consistent input resolution.
Maya: "Can we use the original ultrasound images directly?"
Arjun: "The images have different sizes and resolutions, so the model needs a fixed input size."
The BUSI ultrasound images are approximately 500 × 500 pixels on average. The team considered common input resolutions of:
For PU-UNet, they selected 256 × 256 pixels as a balance between computational requirements and the spatial detail needed for segmentation.
Maya: "Why not use a smaller resolution?"
Arjun: "A small lesion can become only a few pixels wide after resizing. That can remove important boundary information."
Maya: "So image resolution can affect segmentation quality."
Arjun: "Exactly."
The selected resolution provided the spatial detail needed to represent relationships between texture, shape and boundaries while keeping the computational requirements manageable.
The team then integrated Product Units into the U-Net architecture.
Their initial approach placed Product Units throughout the network. The results worsened.
That led to another question.
Maya: "Why would adding Product Units in more places make the results worse?"
Arjun: "Because different layers learn different types of information."
Early layers tend to capture lower-level features such as:
Deeper layers capture higher-level representations such as:
The experiments indicated that Product Units were more useful in these deeper layers, where higher-level feature relationships were being represented.
This led to the development of PU-UNet, with Product Units placed strategically within the deeper layers of the U-Net architecture.
The team evaluated PU-UNet using the BUSI Breast Ultrasound Dataset, which contains 780 annotated ultrasound images categorized as normal, benign and malignant.
Maya: "What metrics matter most for segmentation?"
Arjun: "Dice Score and IoU. Both tell us how closely the predicted tumor region matches the expert annotation."
The Dice Similarity Coefficient (DSC) measures the overlap between the predicted segmentation and the reference annotation.
Intersection over Union (IoU) measures the intersection between the predicted and reference regions relative to their combined area.
The evaluation showed:
The results indicate that incorporating Product Units into selected deeper layers can improve the way feature relationships are represented within the segmentation architecture.
Performance metrics provide one view of a model. The team also wanted to examine where the model was focusing when making predictions.
Maya: "Accuracy alone does not tell us what the model is looking at."
Arjun: "That is where explainability becomes useful."
The team used Grad-CAM visualizations to examine model attention.
Grad-CAM generates heatmaps that highlight the image regions contributing to a model's prediction. In this experiment, the visualizations indicated that PU-UNet focused more consistently on tumor regions, with less attention on irrelevant background structures.
The visual results provided an additional perspective on the quantitative evaluation. They showed where the model was focusing and helped the team examine whether its attention aligned with the tumor regions identified in the annotations.
The research started with a practical segmentation problem and led to a series of architectural experiments.
The key observations were:
Together, these observations formed the basis of PU-UNet and the team's exploration of Semantic Engineering in medical image segmentation.
At Accion Labs, Semantic Engineering is part of our approach to building AI-led solutions that work with enterprise knowledge, data and software systems. The same principle of understanding relationships between information can be applied across healthcare and other complex domains where context matters.