Improved unsupervised physics-informed deep learning for intravoxel incoherent motion modeling and evaluation in pancreatic cancer patients.

abstract CC BY

PurposeEarlier work showed that IVIM‐NETorig, an unsupervised physics‐informed deep neural network, was faster and more accurate than other state‐of‐the‐art intravoxel‐incoherent motion (IVIM) fitting approaches to diffusion‐weighted imaging (DWI). This study presents a substantially improved version, IVIM‐NEToptim, and characterizes its superior performance in pancreatic cancer patients.MethodIn simulations (signal‐to‐noise ratio [SNR] = 20), the accuracy, independence, and consistency of IVIM‐NET were evaluated for combinations of hyperparameters (fit S0, constraints, network architecture, number of hidden layers, dropout, batch normalization, learning rate), by calculating the normalized root‐mean‐square error (NRMSE), Spearman’s ρ, and the coefficient of variation (CVNET), respectively. The best performing network, IVIM‐NEToptim was compared to least squares (LS) and a Bayesian approach at different SNRs. IVIM‐NEToptim’s performance was evaluated in an independent dataset of 23 patients with pancreatic ductal adenocarcinoma. Fourteen of the patients received no treatment between two repeated scan sessions and nine received chemoradiotherapy between the repeated sessions. Intersession within‐subject standard deviations (wSD) and treatment‐induced changes were assessed.ResultsIn simulations (SNR = 20), IVIM‐NEToptim outperformed IVIM‐NETorig in accuracy (NRMSE(D) = 0.177 vs 0.196; NMRSE(f) = 0.220 vs 0.267; NMRSE(D*) = 0.386 vs 0.393), independence (ρ(D*, f) = 0.22 vs 0.74), and consistency (CVNET(D) = 0.013 vs 0.104; CVNET(f) = 0.020 vs 0.054; CVNET(D*) = 0.036 vs 0.110). IVIM‐NEToptim showed superior performance to the LS and Bayesian approaches at SNRs < 50. In vivo, IVIM‐NEToptim showed significantly less noisy parameter maps with lower wSD for D and f than the alternatives. In the treated cohort, IVIM‐NEToptim detected the most individual patients with significant parameter changes compared to day‐to‐day variations.ConclusionIVIM‐NEToptim is recommended for accurate, informative, and consistent IVIM fitting to DWI data.

introduction CC BY

The intravoxel incoherent motion (IVIM) model1 for diffusion‐weighted imaging (DWI) shows great potential for estimating predictive and prognostic cancer imaging biomarkers.2, 3, 4, 5 In the IVIM model, DWI signal is described by a bi‐exponential decay, of which one component is attributed to conventional molecular diffusion and the other to the incoherent bulk motion of water molecules, typically credited to capillary blood flow. Hence, IVIM simultaneously provides information on diffusion (D; diffusion coefficient), capillary microcirculation (D*; pseudo‐diffusion coefficient), and the perfusion fraction (f) without the use of a contrast agent.6, 7, 8 However, despite IVIM’s great potential,2, 3, 4, 5 it is rarely used clinically. Two major hurdles preventing routine clinical use of IVIM are its poor image quality and the long fitting time.9, 10, 11 Tackling these shortcomings will help toward wider use of IVIM.12 Currently, IVIM is often fitted using the conventional least squares (LS) algorithm. However, more accurate alternative approaches have been suggested.9 Until recently, Bayesian algorithms for IVIM fitting13 were most promising regarding inter‐subject variability,9 precision, accuracy,14 and smooth parameter maps, suggesting less noise.15 Conversely, Bayesian approaches are substantially slower (9 × 10−2 s/vox11) than the already slow LS approach (8 × 10−3 s/vox11). Furthermore, Bayesian approaches may lead to biased perfusion estimates of the IVIM model.16 Recently, a promising alternative for IVIM fitting was introduced: estimating IVIM parameters with deep neural networks (DNNs). Initially, Bertleff et al17 introduced a supervised DNN for IVIM parameter estimation, in which the network was trained on simulated data for which the underlying parameters were known. However, the strong assumption of simulated training and test data being identically distributed could limit the network’s performance in vivo, where noise behaves less ordered. We solved this shortcoming in earlier work,11 where we used unsupervised physics‐informed deep neural networks (PI‐DNNs).18, 19 PI‐DNNs formulate a physics‐informed‐loss‐function that finds learned parameters through an iterative process. In this case, the PI‐DNN used consistency between the predicted signal from the IVIM model and the measured signal as a loss term in the DNN. This resulted in an unsupervised PI‐DNN capable of training directly on patient data with no ground truth: IVIM‐NETorig. We demonstrated in both simulations and volunteer analysis that IVIM‐NETorig is superior to the conventional LS approach and even performs (marginally) better than the Bayesian approach. Furthermore, IVIM‐NETorig’s fitting times were substantially lower (4 × 10−6 s/vox11) than the LS and Bayesian approaches. However, that proof of principle IVIM‐NET study did not explore many hyperparameters and focused on volunteer data. In this work, we hypothesize that IVIM‐NETorig can be improved by exploring the architecture of the network, its training features and other hyperparameters. Hence, we characterized the performance of IVIM‐NET for different hyperparameter settings by assessing the accuracy, independence, and consistency of the estimated IVIM parameters in simulated IVIM data. Finally, we compared the performance of our optimized IVIM‐NET to the LS approach and a Bayesian approach in patients with pancreatic ductal adenocarcinoma (PDAC) receiving neoadjuvant chemoradiotherapy (CRT) in terms of image quality, parameter to noise ratio, test‐retest reproducibility, and sensitivity to treatment effects.

ivim‐net CC BY

We initially implemented the original PI‐DNN (IVIM‐NETorig)11 in Python 3.8 using PyTorch 0.4.1.20 The input layer consisted of neurons that took the normalized DWI signal S(b)/S(b = 0) as input, where S(b) is the measured signal at diffusion weighting b (b value). The input layer was followed by three fully connected hidden layers. Each hidden layer had several neurons equal to the number of measurements (b values and the number of repeated measures) and each neuron, in turn, contained an exponential linear unit activation function.21 The output layer of the network consisted of the three IVIM parameters (D, f, D*). To enforce the output layer to predict these IVIM parameters, two steps were taken. First, the absolute activation function was taken of the neuron’s output (X) to constrain the predicted parameters, for example, to compute D:(1)D=X1. Second, a physics‐based loss function was introduced that computed the mean squared error between the measured input signal, S(b), and the predicted IVIM signal Snet(b), which was obtained by inserting the predicted output parameters into the normalized IVIM model. Hence:(2)L=1B∑b∈BSbSb=0‐Snetb2,with(3)Snetb=fe‐bD∗+1‐fe‐bD,where B is the total number of image acquisitions. Next, we evaluated whether seven novel hyperparameters (Table 1; Figure 1) of IVIM‐NET improved fitting results. First, instead of fixing S0, we added S0 as an additional output parameter, to allow the system to correct for noise in S(b = 0). Second, to restrict parameter values to physiologically plausible ranges, scaled sigmoid activation functions instead of absolute activation functions were used to constrain the predicted parameters (Table 1), for example, to compute D:(4)D=Dmin+sigmoidX1∗Dmax‐Dmin,where D min and D max are the fit boundaries. Bound intervals of the sigmoid activation functions were chosen 60% wider (30% at each side) than the fit boundaries of the LS and Bayesian approaches (specified in section 2.2 below) to compensate for decreasing gradients at the asymptotes of the sigmoid function. Third, we varied the number of hidden layers between one and nine. Fourth, we used dropout regularization22 in all hidden layers except for the last one. Dropout randomly removes a set percentage of network‐weights each iteration during training. Fifth, we used batch normalization,23 which normalizes the input by re‐centering and re‐scaling, and, consequently, preserves the representation ability of the network. Sixth, to reduce unwanted correlation between estimated parameter values, we implemented an alternative network architecture in which parameter values were predicted, in parallel, by independent sub‐networks (Table 1; Figure 1). Furthermore, we evaluated different learning rates (LR) of the Adam optimizer,24 ranging from 1 × 10−5 to 3 × 10−2, and with constant β = (0.9, 0.999). In traditional deep learning, training and evaluation are done on separate datasets, but as this is an unsupervised DNN approach, training was done on the same data as evaluation.11, 25 So, for simulations, these were simulated data, and in vivo, these were in vivo data. 90% of the data were used for training, and 10% of the data were used for validation. Early stopping occurred when the validation loss did not improve over 10 consecutive training epochs. Given the large amount of training data and the limited number of network parameters, each epoch consisted of only 500 random batches. So, effectively the network saw 500 × 128 IVIM curves in between validations.

simulations: characterization and optimization CC BY

100,000 IVIM curves were simulated to investigate the effects of different hyperparameters on the accuracy, independence, and consistency of the estimated IVIM parameters. DWI signals were simulated based on Equation 3 with S 0 = 1, 11 b values (b = 0, 5, 10, 20, 30, 40, 60, 150, 300, 500, and 700 s/mm2), and pseudorandom uniformly sampled values of D: 0.5 × 10−3 to 3 × 10−3 mm2/s, f: 5 to 55%, and D*: 10 × 10−3 to 100 × 10−3 mm2/s. These ranges were slightly broader than the typical values found in abdominal IVIM.26 Random Rician noise in the form of complex Gaussian noise was added to the curves with predefined signal‐to‐noise ratio (SNR) levels (constant noise amplitude over b values; SNR defined at b = 0 as S(b = 0)/σ, with σ the SD of the Gaussians).27 Accuracy was assessed as the normalized root‐mean‐square error (NRMSE) between the ground truth parameter values and the estimated IVIM parameters. Independence of the parameter estimates was assessed by the Spearman rank correlation coefficients (ρ) between all parameter pairs. As the simulated data were independent and random, a ρ should be 0. The absolute value of ρ was taken, as both positive and negative deviations from zero are equally undesirable. Some networks always returned the same value for D*, independent of the input data (Supporting Information Figure S1, which is available online). For such cases, ρ is technically undefined. As these cases are undesirable ρ was set to 1. As training a DNN is a stochastic process, training on the same dataset results in different final network‐weights, and consequently, different predictions on the same data. To assess the consistency of estimated parameter values, each network variant was trained 50 times on identical data, where each repeat had a new random initialization, dropout, and batch selection. The normalized coefficient of variation per parameter over the repeated trainings (CVNET) was taken as a measure of the consistency:(5)CVNET=1x¯true1n×m‐1∑i=1n∑j=1mxi,j‐x¯i2,where x̄true is the mean simulated IVIM parameter value, n the number of simulated curves, m the number of repeated trainings, xi,j is the j th repeated prediction of the i th simulated decay curve, x̄i is the mean over the repeated m predictions of the i th simulated signal curve. As the LS and Bayesian approaches are deterministic, their CVNET was zero. As a result of the repeated training, we obtained 50 values for the NRMSEs and ρ’s. Therefore, the median and interquartile ranges were reported. As a baseline for comparison, we evaluated the IVIM parameters (D, f, D*) in IVIM‐NETorig, the LS and Bayesian approaches. We used the Levenberg‐Marquardt non‐linear algorithm for the LS fit.28, 29 For the Bayesian approach, we used the algorithm from previous work.11 For both the LS and Bayesian approaches, S0 was included as a fit parameter. The Bayesian approach used a data‐driven lognormal prior for D and D*, and a beta distribution for f and S0. The prior distributions were determined empirically by fitting these distributions to the results from the LS approach on the same dataset. The maximum a posteriori probability was used as an estimate of the IVIM parameters. The LS and Bayesian approaches were performed with fit boundaries of D: 0 × 10−3 to 5 × 10−3 mm2/s, f: 0 to 70%, D*: 5 × 10−3 to 300 × 10−3 mm2/s, and S0: 0.7 to 1.3. After baseline characterization, IVIM‐NET was optimized by testing various combinations of the hyperparameters (Table 1; Figure 1). Previous studies reported reliable SNR values of IVIM in the abdomen between 10 and 40.30, 31, 32, 33 So, to simulate reliable abdominal IVIM signals, an SNR of 20 was chosen for hyperparameter evaluation. We trained the network on the simulated signals using every combination of the following options: fit S0 parameters, absolute or sigmoid constraints, parallel network, dropout, and batch normalization ‐ while fixing the number of hidden layers to three (used in IVIM‐NETorig, Table 1) and the LR to 1 × 10−4. In an exploratory phase, we found that reducing the LR from 1 × 10−3 (IVIM‐NETorig) to 1 × 10−4 was essential for obtaining networks with improvements in accuracy, independence, and consistency. Each network (ie, combination of hyperparameters) received a ranking in each of the nine performance measures (three metrics for three parameters), and these nine ranks were summed. Hence, the best possible summed rank is 9. The best performing network was then chosen by selecting the network with the lowest summed rank. With the best options for the fit S0 parameters, constraints, parallel network, dropout, and batch normalization, we tested the performance of the network as a function of the LR and the number of hidden layers (Table 1). From those results, we finally selected the best performing optimized network by again selecting the lowest summed rank: IVIM‐NEToptim. IVIM‐NEToptim’s performance was then characterized and compared to the LS approach, Bayesian approach and IVIM‐NETorig for SNR values between 8 (low) and 100 (high).

verification in patients with pdac CC BY

We used two IVIM datasets of patients with PDAC to validate IVIM‐NEToptim's performance in vivo: one dataset to assess test‐retest reproducibility, and one to test whether we can detect treatment effects. Both studies were approved by our local medical ethics committee, and all patients gave written informed consent. Both datasets (NCT01995240; NCT01989000) were published earlier.9, 34, 35 The first dataset consists of 14 patients with locally advanced or metastatic PDAC who underwent IVIM in two separate imaging sessions (average 4.5 days apart, range: 1‐8 days) with no treatment in‐between. The second dataset consisted of nine PDAC patients with (borderline) resectable PDAC who received CRT as part of the PREOPANC study36 where patients were scanned before and after CRT. MRI data were acquired using a 3T MRI scanner (Ingenia, Philips, Best, The Netherlands). A respiratory triggered (navigator on liver dome) 2D multi‐slice diffusion‐weighted echo‐planar imaging was used with parameters: repetition time (TR) > 2200 ms (depending on respiration speed), echo time (TE) = 45 ms, flip angle (FA) = 90 deg, field of view (FOV) = 432 × 108 mm2, acquisition matrix = 144 × 34, 18 slices, slice thickness = 3.7 mm and 12 b values (directions): 0 (15), 10 (9), 20 (9), 30 (9), 40 (9), 50 (9), 75 (4), 100 (12), 150 (4), 250 (4), 400 (4), and 600 (16) mm2/s. Fat suppression was carried out with a gradient reversal during slice selection and spectral presaturation with inversion recovery. Diffusion gradient times were 10.1 ms with a delay between diffusion gradients onset of 22.6 ms. DWI images were co‐registered to a reference volume consisting of a mean DWI image over all b values using deformable image registration in Elastix.37 A radiologist (10 y of experience in abdominal radiology) and researcher (4 y of experience in contouring pancreatic cancer) drew a region of interest (ROI) in the tumor in consensus. IVIM parameter maps of D, f and D* were derived using the LS approach, Bayesian approach, and IVIM‐NEToptim. Background voxels were removed automatically before fitting by removing voxels with S(b = 0) < 0.5 × median(S(b = 0)). Fitting was done without averaging over the diffusion directions. IVIM‐NEToptim was trained on all combined patient data. Values under 0 for D, f, and D* were considered not physiologic and set to 0, and for further statistics, values of D* were masked where f < 5% as D* only “exists” in perfused voxels. All computations were carried out on a single core of a conventional desktop computer (CPU: Intel Core i7‐8700 CPU at 3.20 GHz). The average fitting time of each algorithm was recorded. First, the parameter maps were compared qualitatively in terms of feature clarity, and by visually assessing consistency of fit to the IVIM signal in pairs of neighboring voxels. For a quantitative comparison, the parameter to noise ratio (PNR) of the parameter maps was estimated in a homogeneous 2D ROI (>20 voxels) in the liver. PNR was defined as mean/STD of the homogenous ROIs and was calculated for each scan separately. We tested whether the difference in PNR between fit approaches was significant using paired t‐tests. To determine clinical usefulness of IVIM‐NET, we investigated whether we could detect changes in parameter values throughout CRT by comparing patients receiving treatment to the baseline test‐retest repeatability. This analysis was performed with the median parameter values from within the ROIs. To evaluate test‐retest repeatability, intersession within‐subject standard deviation (wSD)38 was calculated for each IVIM parameter using the data from the patients with repeated baseline scans. Bland‐Altman plots were plotted for patients from both cohorts. We calculated the 95% confidence intervals (CIs) from the patients with repeated scans at baseline (assuming zero offsets). In the treated cohort, we used a paired t‐test to test whether parameters had significantly changed due to treatment within the cohort (significance level α = 0.05). Furthermore, patients from the treatment cohort were added to the Bland‐Altman plots and individual patients who had changes exceeding the 95% CIs were considered to have significant changes in tumor microstructure.39

simulations: characterization and optimization CC BY

The original network, IVIM‐NETorig, showed substantially lower NRMSE for all estimated parameters than the LS and Bayesian approaches. However, IVIM‐NETorig had strong correlations between D* and f (high ρ(D*, f); Table 2 and Figure 2D), and had considerable CVNET. The NRMSE, ρ, and CVNET for all hyperparameter combinations are shown in the Supporting Information Figures S2‐S9. The summarizing sum of ranks (Supporting Information Figures S5 and S9) allowed us to determine IVIM‐NEToptim (Table 1). IVIM‐NEToptim resolved the high ρ(D*, f) found in IVIM‐NETorig (Table 2; Figure 2D,E) and substantially reduced the NRMSE and CVNET. Single changes away from IVIM‐NEToptim can lead to marginally better NRMSE, lower ρ or lower CVNET (Figure 3), but only at a cost to the other two attributes. It is clear that the reduced ρ(D*, f) cannot be attributed to a single parameter, but was a result of the combination of sigmoid constraints and batch normalization (Supporting Information Figure S3). Adding dropout (10%), fitting S 0 and using our parallel network design decreased the NRMSE, while still having a low ρ(D*, f) (Table 2). Increasing dropout in IVIM‐NEToptim or using a single network architecture resulted in similar NRMSE, however, increased ρ(D*, f) (Figure 3; Supporting Information Figures S2‐S4). Generally, increasing the number of hidden layers resulted in a marginally higher ρ, and lower NRMSE and CVNET. A too high/low LR (Supporting Information Figures S5‐S7) caused higher NRMSEs and less consistency. IVIM‐NEToptim was superior to the LS and Bayesian approaches for SNRs 8‐33. Compared to IVIM‐NETorig, IVIM‐NEToptim was associated with improved NRMSE for f and D at all SNRs (Figure 4). For D*, the networks performed similarly regarding NRMSE, with IVIM‐NEToptim performing slightly better at SNRs > 20 and IVIM‐NETorig for SNRs < 20. IVIM‐NEToptim had lower ρ(D*, f) than IVIM‐NETorig and improved CVNET for all SNR levels.

verification in patients with pdac CC BY

Examples of parameter maps computed with the LS approach, Bayesian approach, and IVIM‐NEToptim together with two individual voxel fits of PDAC patients from both cohorts are presented in Figures 5 and 6. Additional parameter maps of 10 other PDAC patients are shown in the Supporting Information Figures S11‐S20. Qualitatively comparing the parameter maps shows that IVIM‐NEToptim has very similar voxel values as the LS and Bayesian approaches for most voxels. However, where the LS and Bayesian approaches sometimes show “noisy” voxels (ie, different from their neighbors) with substantially higher f (order of 1) and lower D (some voxels as low as D = 0 mm2/s; Supporting Information Figures S11, S12) and D* (oftentimes to the lower bound of D*; Supporting Information Figures S12, S14‐S16, S19), IVIM‐NEToptim often sticks to sensible D, f, and D* that are similar to the neighboring voxels resulting in more homogenous parameter maps. Note that IVIM‐NET is fitted at a per‐voxel level and is unaware of the voxel location. Quantitatively evaluating the parameter maps shows that IVIM‐NEToptim had significantly better PNR than both the LS and Bayesian approaches for D and D* (Table 3) and significantly better PNR than the LS approach for f. In the test‐retest cohort, IVIM‐NEToptim showed the lowest wSD for D and f (Table 3), while the Bayesian approach had the lowest wSD for D*. When averaging IVIM parameters for the repeated patient scans, IVIM‐NEToptim computed a higher D, lower f, and higher D* than the LS and Bayesian approaches in the tumor (Table 3). The repeated scans are visualized as black x’s in the Bland‐Altman plots, together with their 95% CIs in Figure 7. When considering the CRT patients as a whole, IVIM‐NEToptim found a significant increase in mean D and f after treatment, whereas the LS approach found only a significant increase in D after treatment in the tumor (Table 3). The Bayesian approach found no significant change in IVIM parameters. Figure 7 shows the individual change in IVIM parameter values of patients receiving CRT compared to the 95% CIs of the test‐retest cohort. With 10 significant changes, IVIM‐NEToptim detected the most patients with significant parameter changes after CRT, with 4 individual patients with increased D, 3 patients with increased f, and 3 patients with changes in D*. In comparison, the LS and Bayesian approaches detected only two and three significant parameter changes, respectively. The average interference fitting time of IVIM‐NEToptim after training was 3.0 × 10−5 s/vox, whereas the average fitting times of the LS and Bayesian approaches were 4.2 × 10−3 s and 1.0 × 10−1 s/vox, respectively. The median training time for IVIM‐NEToptim (20 repeats) was 572 s with a range of 401 to 685 s, which for our dataset resulted in 2.9 × 10−4 s/vox training.

discussion CC BY

This study is the first to show the potential clinical benefit of DNNs for IVIM fitting to DWI data in a patient cohort. We successfully developed and trained IVIM‐NEToptim, an unsupervised PI‐DNN IVIM fitting approach to DWI data that predicts accurate, independent, and consistent IVIM parameters in simulations and in vivo in patients with PDAC. IVIM‐NEToptim consisted of a parallel network architecture with two hidden layers, batch normalization, dropout of 10%, sigmoid constraints, and fitted S 0. Optimized training was performed using an Adam optimizer with an LR of 3 × 10−5. In simulations, IVIM‐NEToptim outperformed the original version, IVIM‐NETorig, by offering more accurate estimates of D, f and D*, with substantially less correlation between the estimated parameters D* and f, and more consistent parameter prediction. Furthermore, simulations demonstrated that IVIM‐NEToptim had substantially better accuracy than the conventional LS and state‐of‐the‐art Bayesian approaches. Finally, in patients with PDAC, IVIM‐NEToptim also outperformed the alternatives. IVIM‐NEToptim showed the most detailed and significantly less noisy parameter maps, and a significant change in D and f for the whole cohort receiving CRT. Furthermore, IVIM‐NEToptim was associated with the best test‐retest repeatability (smallest wSD) for D and f, which allowed it to detect the most patients with significant changes in all IVIM parameters after CRT. IVIM‐NEToptim detected a significant positive trend in D and f for the whole cohort of patients receiving CRT, whereas the LS approach only found a significant positive trend in D. Also, IVIM‐NEToptim detected four patients with a significant parameter increase for D, whereas the LS approach detected only one patient. These findings strongly suggest IVIM‐NEToptim as a good alternative for IVIM fitting in PDAC patients. Findings from other studies support this increase in D 40 and f 41 during CRT in PDAC patients. In general, PDACs tend to have lower diffusion due to the impeded water movement of compressing cells.42 Furthermore, PDACs are typically hypoperfused due to solid stress generated by the dense stroma and significant tumor sclerosis creating elevated interstitial pressure, which compresses tumor feeding vessels.41, 43, 44 An increase in diffusion is likely a consequence of a reduction in cell density due to necrosis as a result of treatment.45, 46 An increase in perfusion can be explained two‐fold: an effective treatment could reduce the amount of stroma and associated solid stress47, 48; furthermore, cell necrosis inside the tumor can reduce interstitial pressure.47 Not all patients demonstrated a significant change induced by treatment. Therefore, using IVIM to discriminate between individual treatment effects may be feasible in the future. As treatment of these patients was part of induction therapy and patients received surgery directly after, overall survival cannot be attributed purely to CRT effects. Hence, given the limited number of patients and the diluted treatment effect, we did not compare overall survival between patients that showed potential treatment effects and others. Our previous work35 showed that the LS approach for IVIM fitting was sensitive to individual treatment effects. However, the high wSD limited the study to detect individual treatment effects. Furthermore, that work35 used denoised DWI b‐images that substantially degraded image sharpness and tumor boundaries were harder to detect (eg, compare figures from this work to example figures from earlier work9). Conversely, our present study demonstrates that DNNs can estimate parameter maps directly from the noisy data resulting in sharp high‐quality IVIM parameter maps. For most voxels, IVIM‐NEToptim produces very similar estimates to the LS and Bayesian approaches. However, within the tumor ROI, IVIM‐NEToptim shows consistently different mean baseline parameters than the LS and Bayesian approaches (Table 3). We believe that there are two major contributors to this discrepancy in mean values. (1) The LS and Bayesian approaches have more noisy parameter maps with some individual voxels showing extreme estimates. (2) IVIM‐NEToptim is seemingly better at estimating parameters in regions of poor perfusion. The first observation is demonstrated by the individual voxel fits (Figures 5 and 6; Supporting Information Figures S11‐S20), where the LS and Bayesian approaches occasionally compute noisy IVIM parameters with a substantially higher f, and lower D and D* than IVIM‐NEToptim. As the LS approach minimizes the sum of the residuals, this parameter combination could describe better the noisy data. However, inspecting neighboring voxels with, respectively, similar noisy data shows that the LS and Bayesian approaches are inconsistent in producing the same IVIM parameters, whereas IVIM‐NEToptim is more consistent. The second note is especially interesting for PDACs, which are generally hypoperfused.41, 43 Other studies reported an overestimation of perfusion parameters in poorly perfused tissue,49, 50, 51 and indeed, the LS and Bayesian approaches show high and noisy perfusion fractions maps in the PDACs. Conversely, IVIM‐NEToptim shows consistently low and less noisy perfusion in these regions. Another interesting observation is that in Figure 6, IVIM‐NEToptim shows a very similar f map to the LS approach for almost all tissue, suggesting again that there is no bias. Yet, contrary to the LS approach, IVIM‐NEToptim has a homogeneously low perfusion map in the PDAC (ie, Figures 5 and 6; Supporting information Figures S11‐S20). In the absence of ground truth in patients, we must rely on visual assessment of parameter map quality, the ability to detect treatment response, and the simulation results, in order to infer the best performing estimator. Given the evidence provided in this study, we argue that combined these factors rule in favor of IVIM‐NEToptim. Although IVIM‐NET showed consistently better results both in simulations and in vivo, IVIM‐NET predicts different IVIM parameters in repeated training. This causes a new sort of variability that, until now, was not an issue in fitting parameter maps. There may be methods to mitigate this variability. First, when probing treatment response, we would advise using one network such that this additional effect is not different pre‐ and post‐treatment. Second, to reduce the variation, one could consider taking the median prediction from 10 repeated trainings instead. We did so in an exploratory study where we formed 5 groups of 10 networks and showed that the median of 10 networks was substantially more consistent, with CVNET values of 3.2 × 10−3, 4.9 × 10−3, and 9.3 × 10−3 for D, f, and D*, respectively. Having a set of networks will also allow the user to estimate the variation on the predicted parameter. Finally, although we see this additional uncertainty, we would like to stress that it is secondary to the overall error of the LS approach, which is apparent from the fact that in the simulations, all 50 instances of IVIM‐NEToptim had lower NRMSE than the LS approach. IVIM‐NEToptim outperformed IVIM‐NETorig at SNRs 8‐100 and was superior to the LS and Bayesian approaches for SNRs 8‐33 (Figure 4). However, at extremely high SNR (SNR = 100; Figure 4), the LS approach outperformed IVIM‐NET. The Levenberg‐Marquardt algorithm for the LS function is an iterative function that finds a minimum of the squared difference. For a relatively smooth loss landscape and high SNR signal, the LS algorithm is designed to find the correct parameter estimates. However, at low SNR, the LS approach has trouble finding the correct parameters. This occurs either because the loss landscape is no longer smooth and, hence, it gets stuck in a local minimum, or, what we believe is more common, the noise has changed the signal such that the global optimum no longer is nearby the ground truth parameters. Moreover, a DNN consists of a complex system that needs to encompass estimating the IVIM parameters for all voxels. It turns out that having been trained on all voxels enables better estimates for individual voxels at low SNR. We expect that DNNs focus on more consistent minima with parameter values that are more frequently observed. This might be similar to data‐driven Bayesian fitting approaches.15, 52 Conversely, IVIM‐NET seems to reach a maximum accuracy at high SNR. Potentially more complex DNNs that are optimized with simulations done at high SNR could handle the subtle signal changes of the IVIM parameters at these SNRs. However, typical SNR values for IVIM data are <50. Therefore, our findings suggest that using IVIM‐NET instead of the LS and Bayesian approaches for IVIM fitting would be beneficial in a clinical setting. The choice of the hyperparameters for IVIM‐NEToptim was based on an optimal combination of accuracy, independence, and consistency across all IVIM parameters. However, other hyperparameter options may be more appropriate when characterizing an individual IVIM parameter (eg, when an observer is only interested in D and IVIM is only used to correct for perfusion). Supporting Information Figures S2‐S9 can help interested readers select the best network for their purposes. The high dependency between D* and f that appears in IVIM‐NETorig could not be attributed to a single cause. Initially, we expected that this dependency originated in the fully connected shared hidden layers of the original network. However, ρ remains substantial when adding the “parallel network architecture” to IVIM‐NETorig (Supporting Information Figure S10). Using IVIM‐NEToptim and a single network architecture showed slightly worse performance in simulations regarding ρ(D*, f), but still had sufficient accuracy and consistency. The dependencies between the estimated IVIM parameters are not per se specific to unsupervised DNNs. For instance, similar dependencies between D* and D or f were found in a different data‐driven Bayesian fitting approach.9 For IVIM‐NEToptim, these dependencies were small at clinical SNR values and similar to those of the LS approach. Although simulation studies in parameter estimation are extremely valuable as the underlying parameter values are known, they also come with limitations. One limitation is that the noise characterization of real data can be diverse and hard to model. For instance, DWI artifacts caused by motion are not considered in simulations and may affect the results of fitting the IVIM model.53 Another limitation is the underlying assumption that data are perfectly bi‐exponential. In reality, the IVIM model is a simplification and real data will be more complex.

conclusions CC BY

We substantially improved the accuracy, independence, and consistency of both diffusion and perfusion parameters from IVIM‐NET by changing the network architecture and tuning hyperparameters. Our new IVIM‐NEToptim is considerably faster, and computes less noisy and more detailed parameter maps with substantially better test‐retest repeatability for D and f than alternative state‐of‐the‐art fitting methods. Furthermore, IVIM‐NEToptim was able to detect the most individual patients with significant changes in the IVIM parameters throughout CRT. These results strongly suggest using IVIM‐NEToptim for detection of treatment response in individual patients.