Draft of:

Reingold, E. M. & Wainwright, M. J. (1996).  Response bias correction in the 
	Process Dissociation Procedure:  A re-evaluation?  Consciousness and 
	Cognition, 5, 595-603.  

Response Bias Correction in the Process Dissociation Procedure:

A Re-evaluation?

Eyal M. Reingold

Martin J. Wainwright

Abstract

A. Buchner and E. Erdfelder (this volume) provide a commentary on our analysis of response bias correction in the process dissociation procedure. Unfortunately, this commentary fails to address the substantive issues that were raised in M. J. Wainwright and E. M. Reingold (1996). In the present article, we attempt to clarify some of their misrepresentations, and the inconsistency inherent in their position.

In this paper, we reply to the commentary of Buchner and Erdfelder (this volume; henceforth B&E) on our analysis of response bias in the process dissociation procedure (Wainwright and Reingold, 1996; henceforth W&R). Our original analysis of the response bias issue is, in part, a critique of Buchner, Erdfelder and Vaterrodt-Plunnecke’s (1995; henceforth B&E&V-P) multinomial modeling exposition of the process dissociation procedure (henceforth PDP; Jacoby, 1991; Jacoby, Toth, & Yonelinas, 1993). In particular, the framework proposed by B&E&V-P obscures rather than clarifies the relational assumptions between conscious influences and guessing (C-G), as well as between unconscious influences and guessing (U-G). Furthermore, we demonstrated that explicitly considering these relational assumptions leads to additional corrective models that were ignored by B&E&V-P. We specified a general class of corrective models, and illustrated it by deriving three response bias corrections to the PDP. In their response, rather than directly addressing the substantive issues that we raised, B&E focus their attention on what they considered to be more or less subtle problems in our framework. Our framework is deemed to be not only fatally flawed, but also a simple re-formulation of the corrective model presented in B&E&V-P. Unfortunately, B&E seriously misrepresent our analysis. In addition, they revise the framework presented in B&E&V-P, in an attempt to accommodate our points without conceding their validity. We therefore urge the interested reader, rather than relying B&E’s representations, to examine the issues as they appear in B&E&V-P, and in W&R. In this paper, we undertake the clarification of the central issues that were obscured in B&E’s response.

B&E&V-P introduced an exposition of the PDP within the framework of multinomial processing trees. One of the key selling points of this approach is that it provided an estimate of unconscious processes which does not require the specification of a relational assumption between conscious and unconscious processes. This parameter is the conditional probability of unconscious processes given that conscious processes have failed, and it is labelled U_c-. In deriving equations for U_c-, B&E&V-P used a processing tree in which conscious processes were considered at the first level of the tree, followed by unconscious processes at the second level. Although B&E&V-P acknowledged that an alternative ordering of processes exists, they implied that their choice of ordering was arbitrary. In W&R, we provided a complete exposition of the two possible orderings: Ordering 1 in which conscious processes are considered prior to unconscious processes, and Ordering 2 in which unconscious precede conscious processes (see Figures 2a and 2b; W&R). In contrast to B&E&V-P, we demonstrated that the transition from Ordering 1 to Ordering 2 entails a crucial loss of generality. Specifically, using Ordering 1 allows estimates of unconscious and conscious processes to be computed without specifying a relation between the processes. In contrast, in Ordering 2, a relational assumption must be specified before estimates can be computed. We clearly documented that once relational assumptions are specified, both orderings are effective in deriving the three variants of the PDP. However, unless these assumptions are specified, only Ordering 1 leads to identifiable estimates. It is in this sense that the two orderings are not equivalent, and the choice of ordering in B&E&V-P is non-arbitrary.

In their reply, B&E demonstrate, as we previously showed, that with either ordering, imposing suitable restrictions on the parameters will yield the three variants of the PDP (i.e., independence, redundancy, exclusivity). However, not only does their presentation fail to acknowledge our previous treatment of this issue, but it is also misleading in suggesting that we had completely missed this basic point. Their presentation is also misleading in another important respect: it fails to address the issue at hand. The relevant question is whether or not, prior to the imposition of relational assumptions, the two orderings are equivalent. In this regard, B&E’s equations (1) through (6) do not address the relevant issue, because they simply show that restrictions on Ordering 1 parameters may be translated to restrictions on Ordering 2 parameters. (Again, this translation of restrictions simply reinforces our point that constraints must be specified in the first place.) As exposited in W&R, certain cancellations occur when using Ordering 1. It is precisely these cancellations that allow the parameters C and U_c- to be identifiable. In contrast, no fortuitous cancellations occur in Ordering 2 -- meaning that none of the parameters in this equally valid ordering are identifiable. As a consequence, Ordering 1 yields identifiable estimates of conscious and unconscious processes prior to the specification of relational assumptions, whereas Ordering 2 does not yield identifiable estimates without relational assumptions. Neither B&E&V-P nor B&E acknowledge this difference between Ordering 1 and Ordering 2. Indeed, B&E offer little more reason than mere convenience for choosing Ordering 1, and they completely fail to acknowledge the loss of generality when Ordering 2 is used.

Another important issue regarding the ordered treatment of processes in a multinomial tree is whether it constitutes an assumption of temporal ordering of processes. Placing processes at different levels in a tree does not necessarily imply that these processes are sequentially ordered in time. However, when processes are treated in a fixed order, it is all too easy to conceive of them as being temporally ordered in the same way. As a case in point, Buchner and colleagues seem to have fallen into this very trap when incorporating guessing into their multinomial tree. Their placement of guessing processes at the lowest tree level does not necessarily imply that guessing occurs after conscious and unconscious processes. However, this temporal ordering of processes appears to be precisely what Buchner and colleagues have implicitly assumed. Specifically, they assume that guessing processes do not occur unless both conscious and unconscious processes have failed to provide a basis for responding. If either conscious processes act and the item is recollected, or if unconscious processes act and the item is familiar, then guessing is not initiated. In other words, Buchner and colleagues are necessarily committed to a temporal ordering in which guessing follows the completion of conscious and unconscious processes. It is important to note that B&E strongly deny that their multinomial trees incorporate assumptions about temporal orderings of processes (e.g., Footnote 1, B&E). Thus highlighted is one of the major pitfalls of the multinomial exposition of the PDP: it tends to obscure assumptions about the relations between processes. An additional example of such a confusion will be elaborated in the next section.

In W&R, we analyzed the corrective model of B&E&V-P, in which a parameter for guessing was incorporated within the process dissociation procedure. A key detail obscured by their exposition is that this guessing parameter (g_i and g_e, in inclusion and exclusion respectively) is actually a conditional probability. Specifically, this parameter indexes the conditional probability of guessing given that both conscious and unconscious processes have failed. For the sake of clarity, we re-labelled this parameter as G_c-u-, where the subscripts c- and u- indicate that the probability is conditionalized on the failure of both conscious and unconscious processes. In simply adding this guessing parameter to their processing tree, Buchner and his colleagues failed to consider the interactions between guessing and other processes. In order to properly analyze these critical relations within the multinomial framework, it is necessary to consider the complete processing tree, which we presented as Figure 3 in W&R. In addition to the single conditional parameter G_c-u- used by B&E&V-P, this complete tree contains the other three conditional parameters G_c+u+, G_c-u+, and G_c+u-. To specify models within the multinomial framework, it is necessary to link the conditional parameters to the unconditional probability of guessing G, which Buchner and colleagues also failed to consider. This link is afforded by the following equation (presented as equation (20) in W&R):

G = C*U_c+*G_c+u+ + C*(1-U_c+)*G_c+u- + (1-C)*U_c-*G_c-u+ + (1-C)*(1-U_c-)*G_c-u-(1)

We showed that assumptions about the relations between processes can be translated to constraints on these conditional parameters. In particular, Appendix A of W&R contains the re-derivation of three corrective models within this expanded multinomial framework.

In their reply, B&E voice strong objections to "adding" the three conditional parameters G_c+u+, G_c-u+, and G_c+u-, as well as to "assuming" the unconditional probability of guessing G. It is critical to note, however, that we have not "assumed" any of these parameters. To the contrary, whether or not they are explicitly considered, all of these guessing parameters simply exist, and they are subject to certain mathematical relations (for instance, equation 1 above). Our role has been to highlight the existence of these parameters, and to carefully consider the relations among them. In the course of doing so, we demonstrated that even though the proposal of B&E&V-P did not explicitly include these parameters, their model does specify restrictions about such parameters . In particular, with respect to the relations between processes, we discovered an inconsistency between the stated assumptions, and the assumptions that were computationally implemented. On one hand, the assumptions of exclusivity in C-G and U-G stated by

B&E&V-P imply that G = (1-C)*(1-U_c-)*G_c-u-. On the other hand, from their computational procedure, it follows that G = G_c-u-, which is consistent with stochastic independence but clearly inconsistent with exclusivity of C-G and U-G.

In response, B&E begin by acknowledging that they have made the following assumption: guessing processes occur if and only if both conscious and unconscious processes fail to give evidence for responding. Contrary to our claim, they deny that this assumption has any bearing on the conditional probabilities of guessing "old" G_c+u+, G_c-u+, and G_c+u-. Their assertion is based on the distinction between: (a) the mere occurrence of guessing processes, and (b) the probability of guessing "old". B&E clarify that their assumption refers to the occurrence of guessing processes, and not to the probability of guessing "old". On this basis, they conclude that their assumption has no consequences for the conditional probabilities of guessing "old".

Sadly enough, this distinction -- given its irrelevance to the debate -- has the potential to be misleading. First of all, it is quite clear that whenever we speak of guessing, we are referring to the probability of guessing "old". It is only this kind of guessing that has any relevance to the process dissociation equations. For instance, the guessing parameters g_i and g_e used by B&E&V-P, labelled as Gⁱ_c-u- and G^e_c-u- in our notation, are conditional probabilities of guessing "old". Similarly, all of the additional parameters identified by W&R are probabilities of guessing "old". Therefore, the reader ought to be aware that throughout both B&E&V-P and W&R, whenever guessing parameters are explicitly discussed, they refer to a probability of guessing "old". Secondly, this distinction provides no support whatsoever for B&E’s conclusion. In fact, even given this distinction, their assumption still implies that the three conditional probabilities G_c+u+, G_c-u+, and G_c+u- are all zero. To see this fact, start with one half of their assumption: if conscious or unconscious processes act, then guessing processes do not act. Secondly, note that if guessing processes do not act, then trivially, guessing "old" cannot occur either. Putting together this two-part logical chain, we have the following: if conscious or unconscious processes act, then guessing "old" does not occur. Therefore, in each of the three cases when conscious or unconscious processes act, the conditional probabilities of guessing "old" are zero. That is, it follows from the assumptions in B&E&V-P’s model that G_c+u+, G_c-u+, and G_c+u- are all zero, precisely as claimed inW&R. By setting these parameters to zero in equation (1) above, it immediately follows that G = (1-C)*(1-U_c-)*G_c-u-. In fact, this equality can be directly read off the processing tree in B&E&V-P (see their Figure 2), by following the single branch that leads to guessing "old" and multiplying the parameters at each level. Therefore, the irrelevant distinction notwithstanding, Buchner and colleagues have effectively acknowledged the following: they have made assumptions of exclusivity in the C-G and U-G relations.

Although, as we have shown, B&E&V-P assumed exclusivity of C-G and U-G, they did not properly implement these assumptions in their computation of estimates. Rather, they pursued an unorthodox use of baseline. Typically, baseline is designed to be an empirical measure of the unconditional probability of guessing "old" (G). In contrast, B&E&V-P made use of the baseline as an empirical measure of the conditional probability G_c-u-. This non-standard use of baseline was far from clear in their original paper. Furthermore, other than invoking a vague notion of "cognitive equivalence", B&E&V-P provided no justification for this assumption. In W&R, we pointed out that by definition, neither conscious nor unconscious processes operate in the distractor condition. Therefore, baseline is an estimate of the unconditional parameter G. Thus, by equating the baseline with G_c-u-, B&E&V-P are effectively assuming that G = G_c-u-. This assumption is consistent with stochastic independence, but inconsistent with exclusivity in the C-G and U-G relations.

In response to our argument, B&E raise the distinction between the unconditional probability of guessing "old" to a target (G_t), and the unconditional probability of guessing "old" to a distractor (G_d). While granting that they have assumed G_d = G_{c-u- (t)} (where the subscript t denotes that the probability applies to the target condition), they question whether G_d is necessarily equal to G_t. It is important to note that contrary to the intimations of B&E, equating G_t with G_d is not an assumption specific to W&R. In fact, the vast majority of memory studies employing baseline incorporate such an assumption. For example, in implicit memory research, the standard priming measure is obtained by subtracting performance in baseline trials from performance on experimental trials. This computation assumes both that guessing for experimental items equals guessing for baseline items, and that guessing and memory combine additively. It is therefore incumbent upon Buchner and colleagues to explain how all of these previous implementations of baseline are flawed.

To further support their criticism of equating G_t with G_d, B&E point out its apparently "undesirable" consequences (see paragraph 2b, Section II; B&E). In particular, they focus upon the equation G = B = (1-C)*(1-U_c-)*G_c-u-, which arises from assumptions of exclusivity in C-G and U-G. They argue that it is strange that the probability of guessing "old" to distractors (B) should depend on the probabilities of conscious and unconscious processes (C and U_c-), as well as the conditional probability of guessing "old" to a target (G_c-u-). With a bit of thought, however, it becomes clear that this state of affairs is not at all strange. The above equation expresses the probability of guessing "old" as a function of the conditional probability G_c-u-; therefore, it is not surprising that the conditional antecedents (namely C and U_c-) might enter into the equation. What would be surprising, on the other hand, is an equation that expresses the probability of guessing "old" to distractors (B) as a function of the unconditional probability of guessing "old" (G) to targets, as well as C and U_c-. In fact, precisely such an equation pops out of the estimation procedure propounded by Buchner and colleagues. We start with the equation that arises from their assumptions of exclusivity in C-G and U-G (see also Figure 2 in B&E&V-P): that is, G = (1-C)*(1-U_c-)*G_c-u-. Then, using their baseline estimation B = G_c-u-, and re-arranging, we obtain B = G / [(1-C)*(1-U_c-)]. That is, the probability of guessing "old" to distractors (B) is assumed to be a function of the probability of conscious and unconscious processes acting on targets, as well as the unconditional probability of guessing "old" to targets. This is a truly implausible consequence of the baseline estimation assumption made by B&E&V-P.

Thus, as pointed out by W&R, B&E&V-P were inconsistent in their treatment of the C-G and U-G relations. It is unfortunate that B&E were unable to clarify this inconsistency in their reply. Another serious implication of their failure to explicitly consider relational assumptions in their multinomial exposition is that additional response bias corrections of the PDP were neglected. This issue is elaborated in the next section.

In W&R, we advocated a set-theoretic framework for expressing the relations between processes. Using this framework, we developed a set of four equations (see equations 10 and 11, and the baseline equations from footnote 2 in W&R). This set of equations is general, because it does not impose any restrictions on the relations between processes. In order to illustrate these equations, we derived three corrective models from first principles. These models were derived by imposing certain restrictions on the process overlaps in these equations. Depending on which restrictions are imposed, the set of four equations generates different corrective models. For the sake of completeness, we showed in Appendix A of W&R that these models may be re-derived by using an expanded multinomial processing tree. This multinomial framework is a generalization of the B&E&V-P proposal, because it includes the full set of conditional parameters, as well as the unconditional probability G.

It is thus with good reason that we find puzzling B&E’s claim that their corrective model is equally general. To begin, the obvious question poses itself: if their formulation truly is as general as our framework, why didn’t Buchner and his colleagues derive multiple corrections to the PDP, just as we did in W&R? At the very least, Buchner and colleagues did not recognize this "apparent" generality prior to W&R, a point which ought to be explicitly acknowledged. In truth, however, their formulation is not as general as our framework. It is only by considering the full set of conditional and unconditional guessing parameters that the complete class of corrective models can be re-derived within an expanded multinomial framework. Given that B&E&V-P ignored all guessing parameters except for G_c-u-, their formulation cannot possibly be as general. Furthermore, in their reply, B&E express strong objections to including these parameters at all. If Buchner and colleagues wish to construct a truly general multinomial framework (such as that presented in Appendix A of W&R), then they must begin by conceding the necessity of considering other guessing parameters in addition to G_c-u-. To continue, they must abandon their baseline estimation procedure, because this non-standard assumption imposes restrictions on the relations between guessing and other processes. Their "extended model" yields estimates that are numerically identical to the independent guessing model. More strongly, no other corrections can be derived from their set of four equations (equations 7 through 10 in B&E; or equations 9 through 12 in B&E&V-P). In contrast, our standard use of baseline B as a measure of the unconditional G does not impose any restrictions on the relations between guessing and other processes. Consequently, depending on which relational assumptions are specified, our set of four equations generates different corrections for guessing. It is in this sense that our proposal is obviously more general than the single correction method proposed by B&E&V-P. Thus, since process overlaps are obscured by the multinomial framework, Buchner and colleagues failed to recognize the existence of other corrective models.

In W&R, we undertook a preliminary empirical comparison of three proposed corrections by using the data reported in B&E&V-P. Most important is the nature of the comparison that we undertook. Instead of setting off a corrective model against the uncorrected PDP (as did B&E&V-P), we compared different corrections to one another (see also Yonelinas, Regehr, & Jacoby, 1995; Yonelinas and Jacoby, in press). Again, it is critical to recognize that on a priori grounds, a comparison of a corrective model with the uncorrected PDP is not overly informative. With the uncorrected PDP, a pre-condition of computing estimates is that the inclusion and exclusion baselines be equated. Consequently, it is totally inappropriate to apply the PDP to data in which these baselines are experimentally manipulated so as to be unequal. Therefore, that the correction proposed by B&E&V-P performs better than the uncorrected PDP is wholly unremarkable. In fact, given that the PDP's assumptions have been deliberately violated, it would be surprising if any correction were worse.

In contrast, we undertook the more meaningful comparison between corrective models. Although our analysis was limited by the nature of the data, we showed that both the HITS-FA and the additive model are at least as good as if not better than the correction suggested by B&E&V-P. Moreover, Yonelinas and Jacoby (in press) performed a similar analysis on the same data, and they also showed that their dual-process signal detection model performs at least as well as if not better than the B&E&V-P model. In our view, these conclusions about differences between corrective models are more relevant than any type of comparison to the totally uncorrected PDP. Indeed, the power of a test is inconsequential if the appropriate question is not posed in the first place. Furthermore, it should also be emphasized that there exists very little work comparing different corrective models to one another. In this context, B&E are unfounded in summarily dismissing other corrective models as fatally flawed. As pointed out in W&R, future research should concentrate on contrasting various corrective models in the recognition version of PDP, as well as other versions (e.g., stem completion).

In this paper, we demonstrated that despite B&E’s claims to the contrary, the points made by W&R remain valid. At this juncture, it may be worthwhile to briefly summarize our critique of Buchner and colleagues’ multinomial exposition of the PDP:

(a) As we have shown, B&E&V-P exposition of the uncorrected PDP capitalizes on a non-arbitrary ordering of processes in the multinomial tree to achieve an illusion of generality.

(b) Athough the ordered treatment of processes in a multinomial tree does not necessarily imply temporal ordering of processes, it encourages the conception of processes as sequentially ordered in time, thereby directing attention away from the overlap between processes.

(c) The consequences of (b) are serious. They include implicit assumptions regarding the temporal ordering of processes, confusions regarding the C-G and U-G relational assumptions, and the failure to recognize the existence of alternative corrective models.

In W&R, we introduced a general set-theoretic framework for incorporating response bias into the PDP. We illustrated how different relational assumptions lead to specific corrective models. For the sake of completeness, we showed in Appendix A of W&R how the multinomial exposition by Buchner and his colleagues could be expanded by considering the full set of guessing parameters. In their reply, B&E argue that our set-theoretic framework is a simple re-formulation of their multinomial exposition. However, in their exposition, Buchner and his colleagues do not include the additional parameters that are essential to the derivation of alternative corrections. In fact, they strongly oppose the inclusion of these parameters. Thus, the current implementation proposed by Buchner and his colleagues leads to one and only one corrective model. Not surprisingly then, B&E&V-P did not empirically compare their corrective model to alternative correction methods. When other corrections are applied to B&E&V-P’s data (see W&R, and Yonelinas and Jacoby, in press), they perform at least as well if not better than the B&E&V-P correction. Future research should be conducted to rigorously contrast different corrective models.

References

Buchner, A., Erdfelder, E., & Vaterrodt-Plunnecke, B. (1995). Toward unbiased measurement of conscious and unconscious memory processes within the process dissociation framework.

Journal of Experimental Psychology: General, 124, 137-160.

Jacoby, L. L. (1991). A process dissociation framework: Separating automatic from intentional uses of memory. Journal of Memory and Language, 30, 513-541.

Jacoby, L. L., Toth, J. P., & Yonelinas, A. P. (1993). Separating conscious and unconscious influences of memory: Measuring recollection. Journal of Experimental Psychology: General, 122, 139-154.

Wainwright, M. J. & Reingold, E. M. (1996). Response bias correction in the process dissociation procedure: Approaches, assumptions and evaluation. Consciousness and Cognition, 5, 232-254.

Yonelinas, A. P., & Jacoby, L. L. (in press). Response bias and the process dissociation procedure. Journal of Experimental Psychology: General

Yonelinas, A. P., Regehr, G., & Jacoby, L. L. (1995). Correcting for differences in response bias in a dual-process theory of memory. Journal of Memory and Language, 34, 821-835

Author Notes

Preparation of this paper was supported by a grant to Eyal Reingold from the Natural Science and Engineering Research Council of Canada. We wish to thank Elizabeth Bosman her helpful comments on an earlier version of this paper.