The issue of the scientific nature of economic theory is surprisingly undeveloped; or, at least, not fully formulated. To set up the issues at stake, we will follow Samuelson  and later writers in econometrics, e.g., Berndt  . In these terms, the theory concerning some aspect of economic activity, i.e., a model, is expressed by a system of simultaneous equations,
where Y is an n-vector of endogenous variables and Z is an m-vector of exogenous variables. This system is brought to the data by the method of comparative statics, as expressed by the total differentiation of the system (1),
For purposes of estimation, it is usual to assume that (1) is (at least locally) linear, so that (2) can be more compactly expressed as,
where β, γ and δ are appropriately dimensioned matrices, with δU representing errors embodied in the data. The system (3) is termed the structural form of the model. Estimation then proceeds with respect to the reduced form,
where in (4) . The arrays β and γ are then recovered from the estimated π using some version of multi-stage least squares. Depending upon issues of identification, there may be more than one way to do this for an over-identified system and no way to do this for an under-identified system.
For the theory to be “scientific”, the expression of the structure (3) must take a form such that the outcomes for the estimate of π are in some way limited. If the outcome of the estimation is not consistent with such limit(s), then the model has been falsified, i.e., Popper  .
So far, the issues at stake seem straight-forward. Nevertheless, problems arise in terms of what it is about a proposed structure that is hypothesized by the theory; and, given this, just what limits does the hypothesis create for the estimated reduced form? Before providing his answers to these questions, Samuelson (  p. 23) considers what he terms “A Calculus of Qualitative Relations”. Typically the theory does not fully quantify the entries of the structural arrays; instead, it only proposes their sign patterns determining the directions of influence among the variables. Given this, it may be possible to “sign” entries of the reduced form by working through the algebra of computing . For this, the analytic burden relates to showing that the sign pattern of β requires that at least some entries of the sign pattern of β−1 must have particular signs, independent of the magnitudes of the nonzero entries of β. In fact, much of the subsequent literature on what has become known as qualitative analysis treats the special case where γ = I, since if any entries of β−1 can be signed based upon the sign pattern initially hypothesized for β, it is reasonably easy, given whatever the sign pattern of γ is initially proposed to be, to see if any entries of π can be signed. The Monte Carlo approach identified below will readily detect any signable entries of the reduced form, i.e., by showing such entries always have the same sign, based upon the hypothesized sign pattern of the structure. If entries of π can be signed, then the hypothesized sign patterns of the structural arrays are “scientific”, since the hypothesis can be falsified, if the required signs do not show up in the estimated reduced form.
Having said all of this, Samuelson goes on to propose that the “likelihood” of finding that all of the (potentially millions) of terms in the expansions of β’s determinant and (at least some) cofactors would all have the same sign was too small to be taken seriously. Hence, a qualitative analysis of a model’s structure would virtually never yield entries in the reduced form that had to have particularly signs and hence (potentially) lead to the falsification of the structural hypothesis. Instead, Samuelson proposed that for β derived from the second order conditions of a (possibly constrained) optimization problem, the first n entries of β−1’s main diagonal would have to be negative and, for β a stable matrix with non-negative off-diagonal and negative main diagonal entries, β−1 would have all negative entries, e.g., Hale, et al. (  , p. 148). This could be potentially falsified by the sign pattern of the estimated reduced form. A literature on the conditions under which a qualitative analysis would be successful was initiated by Lancaster  and is well-summarized in Hale, et al.  . None of this dispelled Samuelson’s pessimism about the potential for a successful qualitative analysis, although Hale and Lady  is a significant exception. It is fair to say that such analyses are seldom performed and, if so, seldom found to be successful.
A recent literature, Buck and Lady   and Lady and Buck  , has shown that the concept of a qualitative analysis as discussed above was too restrictive. Specifically, it was shown that although no sign in the reduced form could be signed, based upon the sign patterns of the structural arrays; nevertheless, there were (always) restrictions on what sign patterns that the reduced form could take. Given this, if the estimated reduced form took on a sign pattern forbidden by these restrictions, the proposed structural arrays are thereby falsified. As a simple example, for the case that γ = I and β−1 = π, the sign pattern of the estimated reduced form must be such that it is possible that βπ = I and πβ = I. Given this, the entries of the first row of the estimated reduced form π cannot, for example, all be the negative of the corresponding nonzeros proposed for the first column of β. This result holds even if no entries of the reduced form can be signed. If the estimated reduced form does not satisfy this restriction, the structural hypothesis has been falsified. Buck and Lady   and Lady and Buck  describe a Monte Carlo technique for sampling quantitative realizations of the structural arrays and the corresponding reduced form that will identify such restrictions.
Buck and Lady  showed that the scope of restrictions due to a hypothesized qualitative structure was even broader than previously proposed. In particular, they showed that falsifiable restrictions on the reduced form can be imposed by the zero, and perhaps other, restrictions on the structure, independent of the (unrestricted) nonzero entries. As an example, consider the structure γ = I and,
In the above array, the entries marked “?” are nonzero, but may otherwise be of any sign and magnitude. The zero restrictions selected present an irreducible inference structure with only one cycle of inference. The Monte Carlo sampling and derivation of the reduced form for β−1 = π, found only 256 sign patterns out of the possible 65,536 sign patterns that a 4 × 4 array could take on, barring zeros. This result is due to the zero restrictions only and is independent of the nonzero signs and magnitudes of the other entries. Suppose a multistage least squares process were applied to this structure to estimate the nonzero entries. If any but the 256 reduced form sign patterns consistent with the structure were then found in the estimated reduced form, the resulting estimates of the nonzeros in the structure would be impossible, i.e., could not possibly generate a reduced form with the sign pattern found in the estimated reduced form. Consequently, any estimates of the structural nonzeros would be incorrect, since the zero restrictions themselves have been falsified.
One of the examples of falsification due to zero and other restrictions given in Buck and Lady  is the Klein  Model I. This formulation has been a popular example for pedagogical purposes in econometrics, e.g., Berndt  . It has also been used as an example for qualitative analysis, e.g., Maybee and Weiner  and Lady  . Buck and Lady  showed that the sign pattern of the reduced form estimated by Goldberger  and used by Berndt  falsified the zero and other restrictions on the model’s structural form, independent of the signs and magnitudes of the nonzero behavioral entries of the structural arrays. However, this in itself is not a decisive finding. It turns out that Klein’s Model 1 fails the Sargan  overidentification test for two out of three estimated equations.
In this paper we re-specify the original Klein Model I slightly, but enough to cause it to pass the overidentification tests and re-examine the resulting sign pattern on the theory that the model generated an impossible sign pattern because it was not properly identified in the first place. In the next section Klein’s Model I is briefly reiterated. In Section III, we test the original model’s identification strategy and suggest a slightly modified version that at least passes the usual overidentification tests. Nevertheless, we found that the zero and other restrictions on the structural arrays were again falsified by the sign pattern of the reduced form estimated for this case. Therefore, we suggest that the Monte Carlo sign pattern test presented in Buck and Lady (2016) is a complement to rather than a substitute for standard overidentification tests. Unlike Buck and Lady (2016), in the Appendix we go on to find the particular feature of the reduced form sign pattern that falsify the structural arrays and write out the expansions of the corresponding cofactors of Klein’s coefficient matrix that confirm this result. Conclusions are provided in Section 4.
2. Klein’s Model I
With the error vector not included, the structural arrays for Klein’s Model I are given below, , where
In this structure the endogenous variables are: private consumption (C), investment (I), the private wage bill (W1), income (Y), profits or nonwage income (P), the sum of private and government wages (W), and private product (E); the exogenous variables are the government wage bill (W2), lagged profits (P−1), end of last period capital stock (K−1), lagged private product (E−1), years since 1931 (Year), taxes (TX), and government consumption (G). The entries notated ai, bj and ck are nine in number and represent the behavioral entries of the model that (ultimately) would be estimated via multi-stage least squares. The remaining entries in the structure are restrictions. The sign patterns proposed by Klein for the structural arrays are given by,
An estimate of the reduced form for this model was provided by Goldberger  . The sign pattern of this estimated reduced form is given below,
Significantly, Berndt  , used the Goldberger  reduced form estimates for pedagogical purposes to demonstrate seven different multi-stage least squares methods to estimate the behavioral coefficients. Unknown to both Goldberger and Berndt, the zero and other restrictions on the structure are falsified by this sign pattern, independent of any signs or magnitudes assigned by whatever methods to the behavioral coefficients. In particular, Buck and Lady  reported that for the structure postulated as below,
there did not exist nonzero signs and magnitudes that could be given to the structural entries marked “?” in the above that would result in the sign pattern of the estimated reduced form. As a result, all of the estimates given in Berndt  are impossible.
Thus, Klein Model I generates a sign pattern that is impossible given the sign pattern of the reduced form equations. However, could this sign pattern be a feature of the way in which the model is specified? If the model is not properly identified, then it is not a proper reduction from the reduced form. See Hendry (  , 421-434). In that case, the impossible sign pattern may be the result of an improper identification strategy. If so, then it might be possible to re-specify the model such that it is properly identified and thereby generate a feasible sign pattern. In the following section we test the identification strategy of the original Klein model, find it wanting, and specify a slightly different version of it that at least appears to be a valid reduction from the reduced form.
3. Is Klein Model I Properly Identified?
The Klein Model I is specified, for estimation purposes as follows (Goldberger 1964, 303-304). The variables are defined above.
Klein Model I
Estimating the three behavioral equations using two-stage least squares yields the results shown in Table 1.
As noted above, this model has been shown to be impossible given the reduced form equations. One problem with the model, an issue that might be responsible for this unfortunate result, is that this model is not properly specified. The Sargan  test for overidentification is significant for both the consumption equation (p = 0.0214) and the private wage bill equation (p = 0.0015). These tests indicate that these equations are not valid reductions from the reduced form. Could this be the cause of the embarrassing impossibility result found above?
A bit of experimentation yields the following alternative model, which we call the Klein-Lady-Moody model (KLM). There are only two small changes in behavioral Equation (2) and Equation (3). In the investment Equation (2’) we have added lagged wages, on the theory that, for example, a rise in wages in the previous year could impinge on investment in the current year. In the private sector wage bill equation we have added taxes, which could reduce employment and/or wages in that sector.
Using the above notation, the amended model is expressed by,
In writing out the expansions of the entries of π referred to in falsifying the KLM version of the model given in the Appendix, the above notation is used.
The resulting estimated KLM behavioral equations are presented in Table 2.
Table 1. Klein model I behavioral equations.
Note: t-ratios in parenthesis; *p < 0.05; **p < 0.01.
Table 2. KLM model behavioral equations.
Note: t-ratios in parenthesis; *p < 0.05; **p < 0.01.
The Sargan overidentification test statistics are no longer significant: consumption (p = 0.0782), investment (p = 0.0993), private wages (p = 0.1117). Thus, whatever else might be said about this model, it appears to be a valid reduction from the reduced form. The resulting matrix representation of the KLM model is as follows.
Does this solve the impossibility issue? We use the same Monte Carlo methodology reported on in Buck and Lady  to determine whether this sign pattern is possible. A version of the computer application that implements the Monte Carlo that we used is available at https://astro.temple.edu/~gmlady/RF_Finder/Finder_Page.htm which also contains the Stata programs and data used to estimate the models presented herein.
The reduced form estimated for the KLM model is given below in Table 3.
The Monte Carlo method takes quantitative samples of the structural arrays with the sign patterns given above, computes , and then checks to see if the resulting sign pattern of π as computed from the sample corresponds to the sign pattern of the estimated reduced form given above. The sample is constructed as follows.
The Monte Carlo sampling procedure takes the sign patterns proposed for the arrays beta and gamma and quantifies the arrays consistent with the given sign patterns. The resulting quantifications are then used to compute pi, the reduced
Table 3. KLM estimated reduced form.
form, the sign pattern of which is then compared to the sign pattern of the estimated reduced form. Call the sign patterns proposed for beta and gamma the text-files and the particular quantification chosen the sample-files. The steps in selecting the sample-files from the text-files are as follows.
1) The text files for β and γ are read into their corresponding sample arrays.
2) All entries in the sample-files are set equal to zero if the corresponding entries in the text-files are equal to zero, preserving the zero restrictions.
3) The value of each remaining (not set to zero) entry of the sample-files are chosen individually and randomly from the open interval (0, 100) using a uniform distribution.
4) The values in the sample-files chosen in step 2) above that correspond to negative entries in the text-files are set negative.
5) The value of γ(1, 1) in the sample file is set equal to the negative of β(1, 3) in the sample-file, a restriction of both the Klein Model I and the KLM structural form.
6) All entries other than β(1, 1) that are equal to “1” or “−1” in the structural form are set equal to the absolute value of the value with chosen for beta (1, 1) in the sample file, keeping their appropriate sign as given in the text-files.1
1This produces a sample of coefficients that are consistent with the input text files. However, the values of 1 and −1 (indicating normalization and identities) have all been changed to the random value chosen for β(1, 1). To restore these values to unit values, one could multiply all rows by abs (β (1/(1, 1)), chosen in step 2) above. This sets all entries in the sample-files that are equal to “1” or “−1” in the structural form equal to the same value, scaling all other entries in the sample files as appropriate to maintain the ratios of all entries in the sample files. However, since this is an elementary row operation it does not change the sign pattern of the reduced form achieved by using the sample-files that result by step 6). As a result, the actual sampling algorithm used does not utilize this step.
The sample-files are then used to compute the implied reduced form, the sign pattern of which is compared to the sign pattern of the estimated reduced form. Subject to the precision of the real variables utilized by the computer software being used (Visual Basic Version 6, service pack 6), the sample-files would in principle allow any possible quantification of the text-files with the given sign patterns and structural form restrictions.
This issue is entirely resolved, as shown below, by documenting why a reduced form sign pattern is not found by examination of the expansions of the arrays used in calculating entries of the reduced form.
The sign pattern of the KLM estimated reduced form was never found after investigating millions of possibilities. The software reported that the sign patterns of the sixth column and the seventh row in the estimated reduced form were never found, independent of the rest. Further analysis showed that in the sixth column, the signs of the triplet of entries: , and , were never found, independent of the rest. Also, the signs of the triplet of entries , and , where never found, independent of the rest.
The chance that the Monte Carlo sampling procedure “missed” a sample which would have led to the estimated reduced form sign pattern is vanishingly small for the large number of samples generated; however, the probability of this happening is nonzero. Accordingly, for the first triplet above we wrote out the expansions the cofactors of β times entries of γ appropriate to the computation of these entries of . Having done this, we found that,
The derivation of this result is given in the Appendix. Further study revealed that this result could be readily derived from the system of equations. For , the result found can be expressed by the condition,
From Equation (7),
and from Equation (4),
Substituting this into the relationship from Equation (7) yields,
the result derived from the expansions involved in the computation of . Accordingly, this condition can only be satisfied if equals (at least) one of ( , ). Thus, the signs of this triplet of entries in the estimated reduced form falsifies the model, i.e., are impossible. Further, this result is based entirely upon the accounting relationships (4) and (7) and is therefore independent of the signs and values of the behavioral coefficients. Thus, although our amendments to Klein’s original specification solved the overidentification problem, the model is nevertheless falsified and is falsified independent of the signs and values of the behavioral coefficients.
The theme of this paper is that qualitative analysis and its potential to falsify a structural form based upon its sign pattern and the sign pattern of the estimated reduced form is a tool for model development as well as model dismissal. The demonstration presented here was to react to the falsification of the original Klein Model I, as presented in Buck and Lady  , and test the model for overidentification, which we found was present. Given this, we introduced some modest amendments to the original model, yielding our version, the KLM model. These amendments resolved the overidentification problem, but further analysis revealed that, nevertheless, the KLM model was also falsified by the estimated reduced form sign pattern. Significantly, the model was falsified based upon the zero and other restrictions, with the finding independent of either the signs or magnitudes of the behavioral coefficients of the model, i.e., the falsification was based upon accounting relationships embodied in the model’s structural forms.
Our point is then that such a finding should prompt additional analysis in the further development of the model’s structural form and the underlying data upon which the model is based. The important point is that, among other tests, a qualitative analysis could provide a useful analytical tool to support development. The Monte Carlo technique utilized for this analysis is easily programmed into a spread sheet and can readily reveal what reduced form sign patterns (appear) possible, among other things based only upon the structural form’s restrictions. In our opinion, when these restrictions are falsified by the estimated reduced form sign pattern, further development is appropriate and initiating the second stage of two stage least squares is not.
Appendix: Derivation of Falsification of the KLM Model
In Section III above it was reported that the triplet of signs in the estimated reduced form of the KLM version of Klein’s model I,
, , and ,
was not found by the Monte Carlo methodology in millions of samples. The expansions of the underlying computation of these entries of the reduced form calculated as revealed that,
As a result, the sign pattern found in the estimated reduced form was not possible, since must equal (at least) one of ( , ) for this equality to hold. This Appendix presents the derivation of that result, using the notation for the KLM structural form given in Section 3.
The details of the computation of these three entries of the reduced form are given by,
where the are the appropriate entries in the adjoint of β.
The expansions of the cofactors that correspond to these entries in the adjoint of β are given by:
Based on these expansions, the reader can quickly verify that,
Significantly, this result is independent of the signs and magnitudes of the behavioral coefficients of the KLM model.